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Abstract 



In this paper we study the consistency of different bootstrap procedures 
for constructing confidence intervals (CIs) for the unique jump discontinuity 
(change-point) in an otherwise smooth regression function in a stochastic de- 
sign setting. This problem exhibits nonstandard asymptotics and we argue 
that the standard bootstrap procedures in regression fail to provide valid 
confidence intervals for the change-point. We propose a version of smoothed 
bootstrap, illustrate its remarkable finite sample performance in our simu- 
lation study, and prove the consistency of the procedure. The m out of n 
bootstrap procedure is also considered and shown to be consistent. We also 
provide sufficient conditions for any bootstrap procedure to be consistent in 
this scenario. 

1 Introduction 

Change-point models may arise when a stochastic system is subject to sudden 
external influences and are encountered in almost every field of science. In the 
simplest form the model considers a random vector X = [Y, Z) satisfying the 
following relation: 



where Z is a continuous random variable, ao f3o E M, Co ^ fe] C M and e is 
a continuous random variable, independent of Z with zero expectation and finite 
variance > 0. The parameter of interest is (o, the change-point. 
Despite its simplicity, model (1) captures the inherent "non-standard" nature of 
the problem: The least squares estimator of the change-point Co converges at a 
rate of to a minimizer of a two-sided, compound Poisson process that de- 
pends crucially on the entire error distribution, the marginal density of Z, among 
other nuisance parameters; see Pons (2003), Kosorok (2008b) (Section 14.5.1, pages 
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271-277) or Koul et al. (2003). Therefore, it is not practical to use this hmiting 
distribution to build CIs for (q. Bootstrap methods bypass the estimation of nui- 
sance parameters and are generally reliable in A/n-convergence problems. In this 
paper we investigate the performance (both theoretically and through simulation) 
of different bootstrap schemes in building CIs for (q. We hope that the analysis 
of the bootstrap procedures employed in this paper will help illustrate the issues 
that arise when the bootstrap is applied in such non-standard problems. 
The problem of estimating a jump-discontinuity (change-point) in an otherwise 
smooth curve has been under study for at least the last forty years. More recently, it 
has been extensively studied in the nonparametric regression and survival analysis 
literature; see for instance Gijbels et al. (1999), Dempfle and Stute (2002), Pons 
(2003), Kosorok and Song (2007), Lan et al. (2009) and the references therein. 
Bootstrap techniques have also been applied in many instances in change point 
models. Diimbgen (1991) proposed asymptotically valid confidence regions for 
the change-point by inverting bootstrap tests in a one-sample problem. Huskova 
and Kirch (2008) considered bootstrap CIs for the change-point of the mean in a 
time series context. Kosorok and Song (2007) use a form of parametric bootstrap 
to estimate the distribution of the estimated change-point in a stochastic design 
regression model that arises in survival analysis. Gijbels et al. (2004), in a slightly 
different setting, suggested a bootstrap procedure for model (1), but did not give 
a complete proof of its validity. 

Our work goes beyond those cited above as follows: We present strong theoretical 
and empirical evidence to suggest the inconsistency of the two most natural boot- 
strap procedures in a regression setup - the usual nonparametric bootstrap (i.e., 
sampling from the empirical cumulative distribution function (ECDF) of {Y,Z), 
often also called as bootstrapping "pairs") and the "residual" bootstrap. The 
bootstrap estimators built by both of these methods are the smallest maximizers 
of certain stochastic processes. We show that these processes do not have any 
weak limit in probability. This fact strongly suggests not only inconsistency but 
also the absence of any weak limit for the bootstrap estimators. In addition, we 
prove that independent sampling from a smooth approximation to the marginal 
of Z and the centered ECDF of the residuals, and the m out of n bootstrap from 
the ECDF of {Y, Z) yield asymptotically valid CIs for (q. The finite sample per- 
formance of the different bootstrap methods shows the superiority of the proposed 
smoothed bootstrap procedure. We also develop a series of convergence results 
which generalize those obtained in Kosorok (2008b) to triangular arrays of ran- 
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dom vectors and can be used to validate the consistency of any bootstrap scheme 
in this setup. Moreover, in the process of achieving this we develop convergence 
results for stochastic processes with a three-dimensional parameter which are con- 
tinuous on the first two arguments and cadlag on the third. In particular, we 
prove a version of the argmax continuous mapping theorem for these processes 
which may be of independent interest (see Section A. 1.1). 

Although we develop our results in the setting of (1), our conclusions have broader 
implications (as discussed in Section 7). They extend immediately to regression 
functions with parametrically specified models on either side of the change-point. 
The smoothed bootstrap procedure can also be modified to work in more general 
nonparametric settings. Gijbels et al. (1999) consider jump-point estimation in the 
more general setup of non-parametric regression and develop two-stage procedures 
to build CI for the change-point. In the second stage of their procedure, they 
localize to a neighborhood of the change-point and reduce the problem to exactly 
that of (1). Lan et al. (2009) consider a two-stage adaptive sampling procedure 
to estimate the jump discontinuity. The second stage of their method relies on an 
approximate CI for the change-point, and the bootstrap methods developed in this 
paper can be immediately used in their context. 

The paper is organized in the following manner: In Section 2 we describe the prob- 
lem in greater detail, introduce the bootstrap schemes and describe the appropriate 
notion of consistency. In Section 3, we prove a series of convergence results that 
generalize those obtained in Kosorok (2008b). These results will constitute the 
general framework under which the bootstrap schemes will be analyzed. In Sec- 
tion 4 we study the inconsistency of the standard bootstrap methods, including 
the ECDF and residual bootstraps. In Section 5 we propose two bootstrap proce- 
dures and show their consistency. We compare the finite sample performance of 
the different bootstrap methods through a simulation study in Section 6. Finally, 
in Section 7 we discuss the consequences of our analysis in more general change- 
point regression models. Additionally, we include an Appendix with the proofs 
and some necessary lemmas and results. 

2 The problem and the bootstrap schemes 

Assume that we are given an i.i.d. sequence of random vectors {X„ = (1^, Zn)}'^^^ 
defined on a probability space {Q, A, P) having a common distribution P satisfying 
(1) for some parameter 6*0 := (ao)/3o)Co) G 6 := U [a,b]. This is a semi- 
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parametric model with an Euclidean parameter 60 and two infinite-dimensional 
parameters - the distributions of Z and e. We are interested in estimating (q, the 
change-point. For technical reasons, we will also assume that P(|ep) < 00. Here, 
and in the remaining of the paper, we take the convention that for any probability 
distribution /i, we will denote the expectation operator by /i(-). In addition, we 
suppose that Z has a uniformly bounded, strictly positive density / (with respect 
to the Lebesgue measure) on [a, b] such that mf\z-(^g\<ri f{z) > k > for some t] > 
and that F{Z < a) A F{Z > b) > 0. For 6 = {a, /3, () g 6, x = {y, z) e write 

■m0{x) := - {y - alzKc - f31z>(;f , (2) 
P„ for the empirical measure defined by Xi, . . . , X„, 

1 

M„ (6) := P„ (me) = — (Y, - alz,<^ + /31z.>c)' , (3) 

1=1 

and M {6) := F{mg). The function M„ is strictly concave in its first two coordi- 
nates but cadlag (right continuous with left limits) in the third; in fact, piecewise 
constant and with n jumps (w.p. 1). Thus, M„ has unique maximizing values of 
a and /3, but an entire interval of maximizers for (. For this reason, we define the 
least squares estimator of 6q to be the maximizer of M„ over G with the smallest 
C, and denote it by 

9n ■■= (a„, /3n, (n) = sargmax {M„(6')} , 

where sargmax stands for the smallest argmax. At this point we would like to 
clarify what we mean by a maximizer: if is a cadlag process on an interval /, 
a point X G / is said to be a maximizer if W {x)y W {x~) = sup {W^(s) : s G /}. 
In the context of our problem, (a, /5, () is a maximizer of M„ if M„ (a, /3, Q V 
M„ {a, P, (-) = sup {M^e) : [d) G 6}. 

The asymptotic properties of this least squares estimator are well known. It is 
shown in Kosorok (2008b), pages 271-277, that y/n{an — «o) = Op (1), y/n0n — 
M = Op (1) and n{(n — Co) = Op (1). It is also shown that the asymptotic 
distribution of n(C„ — Co) is that of the smallest argmax of a two-sided compound 
Poisson process. However, the limiting process depends on the distribution of e 
and the value of the density of Z at Co- Thus, there is no straightforward way to 
build CIs for Co using this limiting distribution. In this connection we investigate 
the performance of bootstrap procedures for constructing CIs for Co- 
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2.1 Bootstrap 

We start with a brief review of the bootstrap. Given a sample W„ = {Wi, W2, ■ ■ ■ , 
Wn} ~ L from an unknown distribution L, suppose that the distribution func- 
tion Hn of some random variable Rn = i?„(W„,L) is of interest; i?„ is usually 
called a root and it can in general be any measurable function of the data and the 
distribution L. The bootstrap method can be broken into three simple steps: 

(i) Construct an estimator L„ of L from W„. 

(ii) Generate W; = {W*, . . . , M/^, J ~ L„ given W„. 

(iii) Estimate if„ by if„, the conditional CDF of i?n(W*,I/„) given W„. 

Let d denote the Prokhorov metric or any other metric metrizing weak convergence 
of probability measures. We say that Hn is weakly consistent if d{Hn, Hn) — )■ 0; 
if Hn has a weak limit H, this is equivalent to Hn converging weakly to H in 
probability. Similarly, Hn is strongly consistent if d{Hn,Hn) -4' 0. 
The choice of Ln mostly considered in the literature is the ECDF. Intuitively, 
an Ln that mimics the essential properties (e.g., smoothness) of the underlying 
distribution L can be expected to perform well. Despite being a good estimator in 
most situations, the ECDF can fail to capture some properties of L that may be 
crucial for the problem under consideration. This is especially true for nonstandard 
problems. In Section 4 we illustrate this phenomenon (the inconsistency of the 
ECDF bootstrap) when n(^n — Co) is the random variable (root) of interest. 
We denote by X = a ((-^n)^i) the a-algebra generated by the sequence (X„)^i 
and write (■) = P(-|j£) and Ex(-) = E(-|je). We approximate the CDF 
of An = n{(n — Co) by Px(A* <x), the conditional distribution function of 
A* = rriniCn ~ Cn) ^ud use this to build a CI for Co, where C^ is the least squares 
estimator of Co obtained from the bootstrap sample. In the following we introduce 
four bootstrap schemes that arise naturally in this problem and investigate their 
consistency properties in Sections 4 and 5. 

Scheme 1 (ECDF bootstrap): Draw a bootstrap sample (l^„*i, Z*^^), . . . , {Y*^^, Z* „) 
from the ECDF of (Fi, Zi), . . . , (Yn, Zn); probably the most widely used bootstrap 
scheme. 

Scheme 2 (Bootstrapping residuals): This is another widely used bootstrap 
procedure in regression models. We first obtain the residuals 
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from the fitted model. Note that these residuals are not guaranteed to have mean 
0, so we work with the centered residuals, e„_i — e„, . . . , e„^„ — e„, where e„ = 
YTj=i^n,j In. Letting denote the empirical measure of the centered residuals, 
we obtain the bootstrap sample {Y*^, Zi), . . . , (Y*^, Z„) as: 

1. Sample e* . . . , e* „ independently from P^. 

2. Fix the predictors Zj, j = 1,. . . ,n, and define the bootstrapped responses 
at Zj as Y*j = anizj<c„ ~^ ^"■'■Zj>c„ + ^nj- 

Scheme 3 (Smoothed bootstrap): Notice that in (1), Z is assumed to have a 
density and it also arises in the limiting distribution of A„. A successful bootstrap 
scheme must mimic this underlying assumption, and we accomplish this in the 
following: 

1. Choose an appropriate nonparametric smoothing procedure (e.g., kernel den- 
sity estimation) to build a distribution Fn with a density /„ such that 

— F||oo and /«—?■/ uniformly on some open interval around Co 
w.p. 1, where / is the density of Z. 

2. Get i.i.d. replicates Z* ^, . . . , Z*^ from Fn and sample, independently, e*^^, . . . , e* 
from P^. 

3. Define ¥*, = <; + /3„1^, ; + e* • for all j = 1, . . . , n. 

Scheme 4 (m out of n bootstrap): A natural alternative to the usual nonpara- 
metric bootstrap (i.e., generating bootstrap samples from the ECDF) considered 
widely in non-regular problems is to use the m out of n bootstrap. We choose 
a nondecreasing sequence of natural numbers {mn}'^=i such that m„ = o(n) and 
rrin — )■ oo and generate the bootstrap sample {Y*^, Z* {),... , (Y*^^, Znm„) from 
the ECDF of {Yi, Zi), . . . , (K„, Z„). Although there are a number of methods avail- 
able for choosing the m„ in applications, there is no satisfactory solution to this 
problem and the obtained CIs usually vary with changing m„. 

We will use the framework established by our convergence theorems in Section 3 to 
prove that schemes 3 and 4 above yield consistent bootstrap procedures for building 
CIs for (q. We will also give strong empirical and theoretical evidence for the in- 
consistency of schemes 1 and 2. Note that schemes 1 and 2 are the two most widely 
used resampling techniques in regression models (see pages 35-36 of Efron (1982); 
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also see Freedman (1981) and Wu (1986)). Thus in this change-point scenario, a 
typical nonstandard problem, we see that the two standard bootstrap approaches 
fail. The failure of the usual bootstrap methods in nonstandard situations is not 
new and has been investigated in the context of M-estimation problems by Bose 
and Chatterjee (2001) and in situations giving rise to n^/^ asymptotics by Abrevaya 
and Huang (2005) and Sen et al. (2010). But the change-point problem considered 
in this paper is indeed quite different from the nonstandard problems considered 
by the above authors - one key distinction being that compound Poisson processes, 
as opposed to Gaussian processes, form the backbone of the asymptotic distribu- 
tions of the estimators - and thus demands an independent investigation. We will 
also see later that the performance of scheme 3 clearly dominates that of the m 
out of n bootstrap procedure (scheme 4), the general recipe proposed in situations 
where the usual bootstrap does not work (see Lee and Pun (1981) for applications 
of the m out of n bootstrap procedure in some nonstandard problems). Also note 
that the performance of the m out of n bootstrap scheme crucially depends on m 
(see e.g., Bickel et al. (1997)) and the choice of this tuning parameter is tricky in 
applications. 



3 A uniform convergence result 

In this section we generalize the results obtained in Kosorok (2008b), pages 271- 
277, to a triangular array of random variables. Consider the triangular array 
{Xn,k = (^n,fc, ^n,fc)}i|fc<„^ defined on a probability space {Q, A, P), where (m„)^^ 
is a nondecreasing sequence of natural numbers such that m„ — )■ oo. Throughout 
the entire paper we will always denote by E the expectation operator with respect 
to P. Furthermore, assume that for each n G N, (X„_i, . . . ,X„„i^) constitutes a 
random sample from an arbitrary bivariate distribution Q„ with Qn{Yni) < ^ ^^"^ 
let Mn{6) := Qni'^e) for all 6 ^ Q, where is defined in (2). Let P be a bivariate 
distribution satisfying (1). Recall that M{9) := F^rriQ) and 9q := sargmax M(6'). 
Let 9n := (a„,/3„,Cn) be given by 

9n = sargmax{Q„(me)}. 

6»ee 

Note that Q„ need not satisfy model (1) with («„, C„). The existence of 9n 
is guaranteed as Qni^e) is a quadratic function in a and (3 (for a fixed () and 
bounded and cadlag as a function in (. For each n, let P* be the empirical measure 
produced by the random sample {Xn,i, ■ ■ ■ , Xn,m„), and define the least squares 
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estimator 9^ = (a* , /?*, C^) G 6 to be the smallest argmax of M*{9) := P* (mg). If 
Q is a signed Borel measure on and is a class of (possibly) complex-valued 
functions defined on write ||Q||^ := sup {|Q(/)| : / G U g: K CR^ 
is a bounded function, write \\g\\K '■= sup^g^ Also, for {z,y) G and 

n G N we write 

in ■■= in (z, y) = y- anl^<c„ - l3nlz>u- (4) 

Let M > be such that \an\ < M for all n. We define the following three classes 
of functions from into M: 

J-' := {1/ (2;) : / C M is an interval} , 

Q ■■= {yfiz):feJ'}U{\y + a\fiz):fe:F,\a\<M}, 

H := {y'f{z):feJ'}. 

In what follows, we will derive conditions on the distributions Q„ that will guar- 
antee consistency and weak convergence of 6**. 

3.1 Consistency and the rate of convergence 

We provide first a consistency result for the least squares estimator, whose proof 
we include in the Appendix (see Section A.2.1). To this end, we consider the 
following set of assumptions: 

(I) ||Q„-P||_^^0, 

(II) ||Q„-P||5^0, 

(III) ||Q„-P||^^0, 

(IV) 9n ^ ^0- 

Proposition 3.1 Assume that (I)- (IV) hold. Then, 9^ 9o. 

To guarantee the right rate of convergence, we need to assume stronger regularity 
conditions. In addition to those of Proposition 3.1, we require the following: 

(V) There are rj, p,L > with the property that for any 6 G (0, r/), there is > 
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such that the following inequalities hold for any n > N: 

, inf I |> ^> | Q«(lcACn<z<cvCn) \ > P, (5) 

L6 

sup {|Q„(e„l^ACn<^<CvcJI} < (6) 
IC-Cn|<52 ^nin 

sup {|Q„(e„lz<CAcJI + |Qn(enlz>cvcJ|} < (7) 
IC-Cn|«52 

We would like to point out some facts about (V). It must be noted that (6) and 
(7) automatically hold in the case where Z and in are independent under Q„ with 
Qn(en) = 0. Also, (5) is easily seen to hold when the Z's, under Q„, have densities 
/„ converging uniformly to / in some neighborhood of (q, where / is the density 
of Z under P; by a consequence of the classical mean value theorem of calculus. 
With the aid of these conditions, Proposition 3.1 and Theorem 3.4.1, page 322, of 
Van der Vaart and Wellner (1996) we can now state and prove (see Section A. 2. 2) 
the rate of convergence result. 

Proposition 3.2 Assume that (I)-(V) hold. Then ^Jm^ia*^ — «„) = Op (1), 
V^(/3: - /3n) = Op (1) and m„(C - Cn) = Op (1). 

Propositions 3.1 and 3.2 provide sufficient conditions on the measures Q„, the 
distribution of each element in the nth row of the triangular array, to achieve the 
same rate of convergence as the original least squares estimators. We would like 
to highlight that we are not assuming that each Q„ satisfy the model (1) with 
(a„,/3„,Cn); all we need is that Q„ and dn approach P and respectively, in a 
suitable manner. 

3.2 Weak Convergence and asymptotic distribution 

We start with some additional set of assumptions: 

(VI) For any function ^/^ : M — > C which is either of the form tl}{x) = e*^^ for some 
^ G M or defined by iplx) = for p = 1, 2, we have: 

mnQn (^(e„)lc„-^<z<c„+^) ^ /(Co)(5 + r/)P(^(e)) V > 0. 
(VII) v^Qn(enlz<c„) ^ and v^Qn(enlz>c„) ^ 0. 

(VIII) nE^„,^ooQn(|en|') < OO. 
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Observe that condition (VI) implies, for all ri,S > 0, and p = 1,2, 



Cn- — <Z<Cn+ — 



^ 0, 



For /i = {h, h, h) G let := e„ + ^) and 

En{h) := m„P; [m^^_^ - m^J . 

We will argue that 

K ■= sargminE„(/i) = {./m^{a*^ - an), y/rri^{P*n - Pn),rnn{Cn - Cn)) 

converges in distribution to the smallest argmax of some process involving two 
independent normal random variables and a two-sided, compound Poisson process 
(independent of the normal variables). 

We derive the asymptotic distribution of the process En and then apply continuous 
mapping techniques to obtain the limiting distribution of h*^. We consider these 
stochastic processes as random elements in the space 'D^-, for a given compact 
rectangle K C M?, of all functions W : K M. having "quadrant limits" (as 
defined in Neuhaus (1971)), being continuous from above (again, in the terminology 
of Neuhaus (1971)) and such that W (■, ■, Q is continuous for all C and W (a, /3, ■) 
is cadlag (right continuous having left limits) for all (a,/3). Write V = Prs. For 
any compact interval / C M let 

A/ = {A : / — i- / |A is strictly increasing, surjective and continuous} 

and write 

log ^'») - 



||A|| := sup 

s^tei s — t 

Then, for any set of the form K = Ax I with A C define the Skorohod topology 
as the topology given by the metric 

dK{^,r):= inf \ sup {|v|/(a,/3,C)-r(a,/3,A(C))|} + 

for r, \1/ G Vk- Endowed with this metric, Vk becomes a Polish space (it is a 
closed subspace of the Polish spaces Dk defined in Neuhaus (1971)) and thus the 
existence of conditional probability distributions for its random elements is ensured 
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(see Dudley (2002), Theorem 10.2.2 page 345). Also, let Vj, I C R, denote the 
space of real valued cadlag functions on /. We refer the reader to Section A.l for 
some results about the Skorohod space. 

We express the process En as the sum of the four terms A„, Cn and Z)„ where 

A4h, h) := 2/11 v^P; (e„l^<^„^(^„+^)) - hlF: (l^<^^^^^^^_^^^) , 
Bn{h2, hs) := 2/i2v^P: (e„l^^^^^(^^^_h3_)) - /i^P; (l^^^^^^^^^^n^y 
Cn{h2, h) := -2m„ ( - + ) P^ (e„l ^^^^^ ) 

- (a„ - /3„ + -|=) P: (lc„+i:^<z<cj 
Z)„(/ii, /13) := -2m„ fy^n - an + (e„l^^<^<^^^_^) 

- m„ - a„ + P:(i 
We define another process := A* + -B* + C* + where 

Alih^) := 2/iiv«':(enlz<cJ-/^?P:(lz<c.; 

5:(/i2) := 2/.2v«':(6„iz>cJ-/^^p:(1z>c„; 

C:(/i3) := -2m„(a„ - /3„)P; (e„l^^^_^^^^<^J 

- m„(a„ - /3n)^P;^ (1. , 
D:(/i3) := -2m„(/3„ - a„)P: (e„l 

- m„(/3„ - aO'P: (lc„<z<c„+^) • 

We work with E'* instead of -En, as their difference approaches uniformly to 
in probability, as shown in the next lemma (proved in Section A. 2. 3), and the 
asymptotic distribution of E*^ is easier to derive. 

Lemma 3.1 Let K C M.^ be a compact rectangle. If conditions (I)- (IV) and (8) 
and (9) hold, then 

Et-Er, 



;<Z<(;n 



Ao. 



K 



P 

Therefore, £"* — En — > as random elements ofVx- In particular, this result is 
true under conditions (I)- (IV) and (VI). 
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As a first step to finding the asymptotic distribution of (E^)'^^^, we show that 
the random sequence is tight in the Skorohod space for any compact rectangle 
K C M^. The proof of the next result is given in Section A. 2. 4. 

Lemma 3.2 Let I C M. be a compact interval and assume that conditions (I)- 
(VIII) hold. Then, the sequence ofM.^ -valued processes 

( y^p;(e„iz<cJ ^ 



\ 



m„P*(e„l^^<2<^^_^ 



(10) 



is uniformly tight in x Vj. Also, if K G M.^ is a compact rectangle, the sequence 
(E*)'^^-^ is uniformly tight in Vk- 

It now suffices to show convergence of the finite-dimensional distributions of the 
processes E*^ to the finite dimensional distributions of some process E* G to 
conclude that i?* converges weakly to E* (and thus En too). With this objective 
in mind, we make the following definitions: Let Zi ~ N (0, cr^P(Z < (^o)) and Z2 ~ 
N (0, (J^P(Z > (^0)) be two independent normal random variables; Vi and z/2 be, 
respectively, left- continuous and right-continuous, homogeneous Poisson processes 



with rate /(Co) > 0; u = {un)'^=i and v 



.1 two sequences of i.i.d. random 



variables having the same distribution as e under P. Assume, in addition, that 
Zi, Z2, z/i, z/2, V and u are all mutually independent. Then, define the process 



:(6)V 



as 



l{t) :- 



\ 



Zi 
Z2 

z/i(-t)lt<o 

^o<i<i/i(-t) '^ilt<o 



and let E* be given by 

E\h) := 2/iiHW(/i3 



hlFiZ <Co) + 2h2E^^\h,)-hl 
+ 2(/3o - ao0'\h;) - («o - /3o)'H(3)(/i3) 
+ 2(«o - m^'\h) - (ao - /3o)'H(5)(/i3) 



>Co 



(12) 
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for h = {hi,h2,hs) G Ml 



We will now prove weak convergence of the sequence of processes {En)'^=i to E*, 
and then use a continuous mapping theorem for the smallest argmax functional (see 
Lemma A. 3) to obtain weak convergence of /i* := sargmax£^„ (/;,). The application 
of Lemma A. 3 requires the weak convergence of processes {En)'^=i to E* and also 
the weak convergence of their associated jump processes. Let S be the class of all 
piecewise constant, cadlag functions : R — )■ M that are continuous on the integers 
with ip{0) = 0; ip has jumps of size 1, and ip{—t) and ip{t) are nondecreasing on 
(0, oo). For an interval I containing in its interior, we write Si = {f \i : / G S}. 
Define the iS-valued (pure jump) 

m = Ut) ■■= ^nP:(ic„+^<z<cJ+"^nP:(ic„<z<c„+^), 

r{t) := z/i(-t)lt<o + z/2(t)lt>o. 

Lemma 3.3 Let I G be a compact interval and K = AxBxIgM.^ a compact 
rectangle. If (I)-(VIII) hold, we have 

(i) S„ S zn X Vj, 

(ii) {El,Jl)^{E\r) znVKxSi, 
(ill) (E„, 4) {E\ J*) m Vk X Si, 
where -w denotes weak convergence. 

For a proof of the convergence result, see Section A. 2. 5. 

To apply the argmax continuous mapping theorem we first show that the the 
smallest argmax of E* is well defined. The proof of the next lemma is provided in 
Section A. 2. 6. 

Lemma 3.4 Consider the process E* defined in (12). Then, for almost every 
sample path of E* , 0* = (0i,02>03) '■— sargmax{E* (h)} is well-defined. Moreover, 
(pl, 02 and 03 are independent; and 0* and 02 are distributed as normal random 
variables with mean and variances a /F{Z < Co) and a^/F{Z > Co), respectively. 

We now state the distributional convergence result for the sequence of least squares 
estimator 0*. For a proof, we refer the reader to Section A. 2. 7. 
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Proposition 3.3 With the notation of Lemma 3.4, if conditions (I)-(VIII) hold, 
then 

\ m„(C-Cn) / 



h* 

"'n 



sargmax{E* (h)} . 



If we take Q„ = P and rrin = n G N, it is easily seen tliat 6n = Oq and conditions 
(I)-(VIII) liold. Hence, we immediately get the following corollary. 

Corollary 3.1 (Asymptotic distribution of the least squares estimators) 

For the least squares estimators {an,f3n,Cn) based on an i.i.d. sequence 
satisfying (1), we have 



n{an - ao), A/n(/3„ - /3o),n(Cn - Co))' ^ sargmax{E* {h)} . 



4 Inconsistency of the bootstrap 

In this section we argue the inconsistency of the two most common bootstrap pro- 
cedures in regression: the ECDF bootstrap (scheme 1) and the residual bootstrap 
(scheme 2). Recall the notation and definitions in the beginning of Section 2. In 
particular, note that we have i.i.d. random vectors {X„ = (F„, Z„)}^^ from (1) 
with parameter 9q defined on a probability space (fi,^, P) and let P„ be the em- 
pirical distribution of the first n data points. We start by stating two results that 
will be used in the sequel. We first show that the least squares estimator 6^ of 9q 
is strongly consistent. This is an improvement of the result obtained in Kosorok 
(2008b) and we refer the reader to Section A. 2.8 for a complete proof. The proof 
of the second lemma can be found in Section A. 2. 9. 

Lemma 4.1 Let K G Q be any compact rectangle. Then, 

(t) ||M„-M||^ ^0, 

(ii) M m Vk, 

(iii) 9n-^9o- 

Lemma 4.2 Let K G be a compact interval and (m„)^]^ be an increasing 
sequence of natural numbers such that rUn — )■ oo and rrin = 0{n). Then, 

p 



(^) ml P„(C„ + £ < ^ < Cn 



for any 7 < 1, and 

K 
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(ii) ml P„ {\tnyi% , i±<-7<; ) ^ for any 7 < 1, and p = 1,2. 
These statements are still true if l;i , {) ^„,t is replaced hyl;, ^v^; , (•) ■ 

We introduce some notation. Let (X, d) be a metric space and consider the X- valued 
random elements V and defined on (fi,^, P). We say that Vn converges 

conditionally in probability to V , almost surely, and write Vn a.s. V ■, if 

Vx{d{Vn,V)>t)^Q Ve>0. (13) 

Similarly, we write Vn'j^ V and say that Vn converges conditionally in probability 
to V , in probability, if the left-hand side of (13) converges in probability to 0. 

4.1 Scheme 1 (Bootstrapping from ECDF) 

Consider the notation and definitions of Section 2.1. To translate this scheme into 
the framework of Propositions 3.1, 3.2 and 3.3, we set m„ = n, Q„ = P„ and 
consider the triangular array {X* ;^ = (^^^fc, ^j^_fc)}i<^<„- Moreover, from Lemma 
4.1 we know that 9n Oq, so we can also take 9n = On- We first prove that the 
bootstrapped estimators converge conditionally in probability to the true value of 
the parameters, almost surely. 

Px 

Proposition 4.1 For the ECDF bootstrap, we have O'^ZI. Oq. 

Proof: Since Y has a second moment under P, it is straightforward to see that 
J-", Q and "H are VC-subgraph classes with integrable envelopes 1, |y| + M and y^, 
respectively. It follows that all these classes are Glivenko-Cantelli and therefore 
conditions (I)-(III) hold w.p. 1. Also, note that, from Lemma 4.1 {iii) condition 
(IV) holds a.s. The result then follows from Proposition 3.1. □ 
Let P* be the ECDF of X*i, . . . ,-^jJ„ and recall the definition of the processes 
An, Bn, Cn, £>n, -E„, A*, 5*, C*, and E*. We then have the following result. 

Lemma 4.3 Let K cM.^ be any compact rectangle. Then 

En-El^Otn Vk. 

Proof: We already know that conditions (I)- (IV) hold w.p. 1 under this bootstrap 
scheme. But Lemma 4.2 implies that (8) and (9) hold in probability. Hence, this 
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result follows by arguing through subsequences and applying Lemma 3.1. □ 
It is evident that condition (VI) doesn't hold in this situation as we know that 

nP„(Co - ^<^<Co + -)-- Poisson(/(Co)(5 + r/)). (14) 
n n 

Hence, we cannot use Proposition 3.3 to derive the limit behavior of /i*. 
We will now argue that E^, and therefore En, does not have any weak limit in 
probability. This statement should be thought in terms of the Prokhorov metric 
(or any other metric metrizing weak convergence on Vk)- If we denote by px 
the Prokhorov metric on the space of probability measures on Vk and by /i„ the 
conditional distribution of E^ given X, to say that (E^)'^^^ has no weak limit in 
probability means that there is no probability measure /i defined on Vk such that 

PK (/^n,/U) > 0. 

The following lemma (proved in Section A. 2. 10) will help us show that the (con- 
ditional) characteristic functions corresponding to the finite dimensional distribu- 
tions of -E* fail to have a limit in probability, which would, in particular, imply 
that E^ does not have a weak limit in probability. 

Lemma 4.4 The following statements hold: 

(i) For any two real numbers s < t, {nP„(^o + ^ < ^ < Co + |)}^]^ does not 
converge in probability. 

(a) There zs /i* > such that for any h> h^:, the sequences 

|nP„(C„ <Z<Cn + ^)}"^^ and {nP„(C„ -^<Z< Cn)}J^^ do not con- 
verge in probability. 

(Hi) For any two real numbers s < t and any measurable function : M — )■ M, 
< nP„(0(F)l/- , jL<2<r„+A) \ does not converge in probability. 

(iv) Let (f) be a measurable function which is either nonnegative or nonpositive 
and such that 0(e + ao) and 0(e + /3o) are nonconstant random variables with 
finite second moment. Then, there is > such that for any h > h^, 
|nP„(0(r)l^^<^<^^+ZL)}^^^ and |nP„(0(F)l^^_z,<2<^j}^^^ do not con- 
verge in probability. 

With the aid of Lemma 4.4 we are now able to state our main result. 

Lemma 4.5 There is a compact rectangle K C M.^ such that neither En nor E^ 
has a weak limit in probability in Vk. 
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Proof: Since Lemma 4.3 and Slutsky's lemma show that En has a weak hmit 
in probabihty if and only if E* has a weak limit in probability, it suffices to argue 
that the statement is true for E^- To prove this, it is enough to show that there is 
some hs such that -E'.*(0, 0, /13) does not converge in distribution. Pick h^, > and 
observe that 



e:(o,o,/i3 



(nK [(2e„-d„ + /3„)l^^ 



converges 



Since — /3„ ao — /^o 7^ we see that i?*(0, 0, /is) will converge weakly 
in probability if and only if A„ := nP* (2e„ — d„ + /3„)1^ <z<c +^ 
weakly in probability. 

The conditional characteristic function of A„ is given by 



(15) 



(e^«^") = ( 1 + -^^n (^(e'«(2e-„ +/?„-&„) _ ^^^^ 



which converges in probability if and only if so does 



. But note that 



nP„ ('(e^«(2>'-/3n-a„) _ 



It is easily seen that (14) and the fact that n(Cn — Co) = Op{l) imply that 



nP„ 1, 



C„<Z<Cn + - 



Op(l). 



Hence, 



nP„ f(emy-^r.-&n) _ 



Cn<^<Cn 



h3 ) -nP„ ((e^?(2^-/3o-«o) _ 1)1 



Cn<^<Cn + 



) 



It follows that Ex (e*^^") has a limit in probability if and only if 



nFn ( (e^«(2i'-/3o-"o) _ 1)1 



Cn<^<Cn 



) 



has a limit in probability. But a necessary condition for the latter to happen is 
that its real part. 



nP„ (^Re(e'«(2y-A3-ao) _ 1)1 
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converges in probability. Since Re(e*^''^^~''°~"°-' — 1) < we can conclude from (iv) 
of Lemma 4.4 that nP„ ^Re(e*^*^^-*^~'^°~"'''' — 1)1^ <z<c +^) ^'^^^ converge in 
probability for all > h^, for some h^, > Q large enough. This in turn implies that, 
for all /i3 > /i*, the conditional characteristic function in (15) does not converge in 
probability and hence -E'*(0, 0, h^) has no weak limit in probability. 
Hence, if K is any compact rectangle containing (0,0, the finite dimensional 
dimensional distributions of E'* on K do not have a weak limit in probability. 
Therefore, i?* does not have a weak limit in probability on □ 



Note that 

( - "n), \fn{f5l - /3„), n(C* - Cn) ) = sargmax | E„(/i) I . 

Thus, the fact that the sequence (-E„)^^ doesn't have a weak limit in probability 
makes the existence of a weak limit in probability for n(^* — C^n) very unlikely. 
However, we do not have the a rigorous mathematical proof this statement. The 
main difficulty in such a proof is that the argmax functional is non-linear and that 
En depends on through indicator functions that do not converge in the limit. 



Remark: It must be noted in this connection that the bootstrap scheme estimates 
the distribution of (A/n(a* — y/n{f3* — /3„)) correctly, and in fact, valid boot- 
strap based inference can be conducted to obtain CIs for ao and Po- This follows 
from the fact that, asymptotically, the maximizers of £'„(■,■, /is) do not depend on 
/13 (see the expressions for A*, _B„, A*, 5*). 

We next provide an alternative additional argument that illustrates the inconsis- 
tency of the ECDF bootstrap. Our approach is similar to that of Kosorok (2008a) 
and relies on the asymptotic unconditional behavior of 



a; := (y^« - ao), v^(/3: - /^o), n(C - Co))- 
For h e R\ we write &n,h ■= Oo + %, ^) and 

En{h) := nP; - mg^ . (16) 
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This corresponds to centering the objective function around 6q. As in (10), we can 
define the processes 

/ y^p;(eiz<co) \ 



si^^t) 
si^^(t) 



v/^P;(e„lz>Co 

'0 + ^ 

nF*(el 



^n(lco-t-^<^<Co; 



fo+^<2<Co) 
^n(lCo<Z<Co+^) 



v 



nP*fel 



(17) 



n\'^^(o<Z<Co+ 



and just as in that case, we can also define the process i?* by 

K{h) := 2h{E.^^\h,)-hl¥l{Z<C,) + 2h^E^^\h,)-hlK{Z>Co) 

+ 2(/3o - ao)^^\h^) - (ao - (^^f^f{h^) 

+ 2(ao - /3o)Hi'H/i3) - («o - /3o)'Hi5H/^3). 

Then, it can be shown that En — E^ — > in for any compact rectangle i^' C M'^ 
and that the sequence {E^)'^^i is tight in V^- 

In what follows we will describe the limiting distribution of E^, namely E*, and 
show that the (unconditional) asymptotic distribution of A* is that of the small- 
est argmax of E*. This result will help us show that the ECDF bootstrap is 
inconsistent. 

We start by introducing some notation. Recall the definitions of the random 
elements Zi, Z2, z^i, 1^2, u and v as in the discussion preceding (11). Also let 
T = (■^n)^i and K = {Kn)'^=i two sequcuces of i.i.d. Poisson(l) random variables. 
Assume, in addition, that Zi, Z2, I'l, 1^2, v, u, r and k are all mutually independent. 



Then, define the process S 



(H«, 
/ 



e(6)v 



as 



Z2 

^0<j<ui(-t) ^jlt<0 

^0<j<u2(t) ^ilt>o 



V ^0<j<U2{t)^3^At>0 j 

for t G M and let E* be given by 

E*{h) = 2hE^'\h)-hlFiZ<Co) + 2h2E^''\h)-hlFiZ>Co) 
+ 2(/3o - ao0'\h) - (ao - Por^^'\h) 
+ 2(«o - Po)^^'\h,) - («o - /3o)'H(5)(/i3) 



(19) 
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for h = {hi, /i2, h^) G M^. Additionally define the tS-valued (pure jump) processes 
J fi , J,^ and J as 

J:it) = Ut) := nP;(l^„+^<^<^J+nP:(l^„<^<^„+i), (20) 
J*{t) := z/i(-t)lt<o + z/2(t)lt>o. (21) 

Lemma 4.6 (proved in Section A. 2. 11) now states the asymptotic distribution of 
En and of n{Q - Co)- 

Lemma 4.6 Consider the processes S„, En, Jn, 2, E* and J* as defined in (17), 
(16), (20), (18), (19) and (21), respectively. Then, unconditionally, 

(i) S„ S m X I?| for any compact interval I C M; 

(ii) {EniJn) ^ {E*,J*) in T>x x Si for any compact interval / C M and any 
compact rectangle i^' = Axi?x/c]R'^; 

(iii) A; = sargmaXy^^^ziEn^h)} -w sargmaXf^^^i{E*{h)} . 

As a consequence, if the ECDF bootstrap is consistent, the variance of sargmaxj^^^A{E*(h)} 
must he twice that of sargmaXf^^^i{E*{h)} . 

As analytic expressions for the asymptotic variances of n{(* — Co) and n{(n — Co) 
are not known, we use simulations to compute them. As an illustration, we take 
e ~ A^(0, 1), Z ~ A^(0, 1), ao = —1, /3o = 1 and Co = in (1). We approximate the 
limiting variances with the sample variances computed from 20,000 observations 
from each of the two asymptotic distributions. Our results are summarized in the 
following table, which immediately shows that the asymptotic variance of n{(n~ Co) 
is not twice that of n(C„ — Co)- Thus the ECDF bootstrap cannot be consistent. 



Random variable 


Asymptotic Variance 


n{Cn - Co) 


7.620948 


n{Cn - Co) 


63.98377 



4.2 Scheme 2 (Bootstrapping "residuals") 

Another resampling procedure that arises naturally in a regression setup is boot- 
strapping "residuals". As with scheme 1, bootstrapping the "residuals" fixing the 
covariates is also inconsistent. Heuristically speaking, the resampling distribution 
fails to approximate the density of the predictor at the change-point Co at rate-n, 
and this leads to the inconsistency. 
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We recall the notation of Section 2. There we described the basic elements of the 
traditional fixed-design bootstrap of residuals and how to compute the bootstrap 
estimates ^* . We first show that these bootstrap estimators converge conditionally 
in probability (almost surely) to the true value of the parameter. Then, we will 
provide a strong argument against the consistency of this bootstrap scheme. For 
notational convenience, we introduce the process /?„ given by 

1 " 

We start by showing that the "centered" empirical distribution for the least squares 
residuals, P^, converges to the distribution of e in total variation distance with 
probability one and its second moment is an almost surely consistent estimator 
of 0"^. This lemma will also be useful for the analysis of the smoothed bootstrap 
procedure. The proof can be found in Section A. 2. 12. 

Lemma 4.7 Let G and (f be, respectively, the distribution and characteristic func- 
tions of e. Then, 



(i) for any rj > we have that sup 



^0; 



(n) ||P^ - G|U ^ 0; 
(ill) J x^dFl^ix) ^ a^; 

(iv) j |x|rfP^(x)^P(|e|); 
(v) if e has a finite third moment under F, then 

lim / |x|^(iP^(x) < oo almost surely. 

n— >oo J 

The next result (proved in Section A. 2. 13) shows that the bootstrapped least 
squares estimators converge conditionally in probability with probability one. 

Proposition 4.2 Let K G Q be a compact rectangle. Then, 

(i) ||i?„ + P*(et)-M„-a2||^,5o; 

(ii) ||i?„ + p;(et)-M-a2||^So; 
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where Mn and M are defined as in (3) and the subsequent paragraph. 
Consider the following process 

Then for n large enough we have that 

( v^(«n - "n), Vn{P*n " /^n), " Cn) ) = sargmax { E„ (/l) I . 

Next we argue that the sequence {En)'^=i does not have a weak limit in probability 
and therefore distributional convergence of their corresponding smallest minimizers 
seems unreasonable. We refer the reader to Section A. 2. 14 for a complete proof of 
the statement. 

Lemma 4.8 There is a compact rectangle K dM? such that the sequence of pro- 
cesses {En)'^=i does not have a weak limit in probability in V^- 

5 Consistent bootstrap procedures 

Here we will prove that the "smoothed bootstrap" (scheme 3) and the m out 
of n bootstrap (scheme 4) procedures yield consistent methods for constructing 
confidence intervals around the parameters. 

5.1 Scheme 3 (Smoothed Bootstrap) 

To show that scheme 3 (smoothed bootstrap + bootstrapping residuals) achieves 
consistency we appeal to Propositions 3.1, 3.2 and 3.3 by proving that the regularity 
conditions (I)- (VIII) of Section 3 hold for this scheme. Recall the description of this 
bootstrap procedure given in Section 2. Let and Fn be the estimated smoothed 
density and distribution function of Z, respectively. For / := [c, c?] C M, a compact 
interval such that Co ^ (c, d), we require the following two properties of /„ and F„: 

\\Fn-F\\u ^ 0; (22) 

^ 0. (23) 

We would want to highlight that these conditions are fulfilled by many density 
estimation procedures. In particular, they hold when the density / is continuous 
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and we let be the kernel density estimator constructed from a suitable choice 
of kernel and bandwidth (e.g., see Silverman (1978)). 

Let 9n = On, rrin = n and Q„ be the distribution that generates the bootstrap 
sample. Observe that under e„ and Z are independent and that Z is a contin- 
uous random variable with density /„. The next result (proved in Section A. 2. 15) 
shows that the bootstrapped least squares estimators achieve the right rate of 
convergence. 

Proposition 5.1 // (22) and (23) hold, then w.p.l, the sequence of conditional 
distributions of ( ^/n{an — ^n), y/^if^n ~ ^n),n{(* — C„) ) is tight. 



Scheme 3 uses an approximation to the density of Z and this turns out to be 
crucial. The bootstrap measures now satisfy property (VI) on Section 3 and the 
bootstrap procedure is strongly consistent, as shown in the next result (proved in 
Section A.2.16). 

Proposition 5.2 For scheme 3, provided that (22) and (23) hold, conditions (I)- 
(VIII) are satisfied with probability one, and thus. 



( y/n{al - an) \ 



sargmax{E* {h)} almost surely. 



5.2 Scheme 4 (m out of n bootstrap) 

For this scheme we will again use the framework established in Section 3. We 
take {mn)'^=i to be any sequence of natural numbers which increases to infinity, 
On = On and Q„ = Pn- The next result (proved in Section A. 2. 17) shows the weak 
consistency of this procedure. 

Proposition 5.3 Ifnin = o{n) and nin — ?■ oo, then conditions (I)-(VIII) hold (in 
probability) and we have 



I y/m^ial - Otn) \ 

y^(/3; - Pn) 



sargmax {E* (h)} in probability. 



(24) 



Remark: To prove Proposition 5.3, we will, in fact, show that for every subse- 
quence (rifc)^]^, there is a further subsequence (n^J^^, such that (I)-(VIII) hold 
w.p. 1 for (nfcj^i and (24) holds almost surely along the subsequence {ukj'^i. 
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6 Simulation experiments 



In this section we report the finite sample performance of the different bootstrap 
schemes on simulated data. We simulated random draws from four different mod- 
els following (1). Each of these corresponded to choosing different pairs {F,G) 
of distributions for Z and e (having mean 0), respectively. The pairs consid- 
ered were (iV(0, 2), iV(0, 1)), (45(4,6) - 2,N{0,1)), (45(4,6) - 2, Unif(-1, 1)), 
and (45(4, 6) — 2, r(4, 2) — 2), where B{-, ■) and r(-, ■) denote the beta and gamma 
distributions respectively. 

For each of these models, we considered 1000 random samples of sizes n = 50, 100, 200, 500. 
For each sample, and for each of the bootstrap schemes, we took An bootstrap repli- 
cates to approximate the bootstrap distribution. The following table provides the 
estimated coverage proportions of nominal 95% CIs and average lengths of the CIs 
obtained using the 4 different bootstrap schemes for each of the four models. 
At this point, we want to make some remarks about the computation of the es- 
timators. We used a kernel density estimator based on the Gaussian kernel and 
chose the bandwidth by the so-called "normal reference rule" (see Scott (1992), 
page 131). In the case of the m out of n bootstrap, we did not use any data driven 
choice of m„, but tried 3 different possibilities: \nt], [n")] and [nit]. We will 
refer to the fixed-design bootstrapping of residuals scheme by FDR. 



Z ~ Af(0,2),e ~ N{0,1) 


Scheme 


n = 50 


n = 200 


n = 500 


Coverage 


Avg Length 


Coverage 


Avg Length 


Coverage 


Avg Length 


ECDF 


0.83 


1.14 


0.79 


0.22 


0.81 


0.08 


Smoothed 


0.94 


0.94 


0.95 


0.19 


0.95 


0.07 


FDR 


0.83 


0.76 


0.86 


0.16 


0.90 


0.06 




0.87 


0.87 


0.91 


0.23 


0.91 


0.08 


|-„9/i0i 


0.85 


1.02 


0.87 


0.21 


0.87 


0.079 


|-„14/15-| 


0.85 


1.05 


0.84 


0.21 


0.86 


0.08 


Z ~ 4B(4,6) - 2, e ~ N{0, 1) 


Scheme 


n = 50 


n = 200 


n = 500 


Coverage 


Avg Length 


Coverage 


Avg Length 


Coverage 


Avg Length 


ECDF 


0.80 


0.54 


0.80 


0.11 


0.81 


0.04 


Smoothed 


0.96 


0.46 


0.94 


0.11 


0.95 


0.47 


FDR 


0.73 


0.32 


0.77 


0.08 


0.79 


0.03 




0.88 


0.53 


0.89 


0.11 


0.90 


0.04 


p„9/i0^ 


0.85 


0.54 


0.86 


0.11 


0.88 


0.04 




0.83 


0.55 


0.84 


0.11 


0.87 


0.04 


Z ~ 4B(4, 6) - 2, e ~ Unif(-1, 1) 


Scheme 


n = 50 


n = 200 


n = 500 


Coverage 


Avg Length 


Coverage 


Avg Length 


Coverage 


Avg Length 


ECDF 


0.80 


0.40 


0.80 


0.08 


0.81 


0.03 


Smoothed 


0.94 


0.33 


0.95 


0.08 


0.96 


0.04 


FDR 


0.75 


0.26 


0.77 


0.06 


0.81 


0.02 




0.88 


0.36 


0.88 


0.09 


0.91 


0.04 


p„9/i0^ 


0.85 


0.39 


0.85 


0.08 


0.87 


0.03 


p„14/15^ 


0.83 


0.39 


0.84 


0.08 


0.85 


0.03 
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Z ~ 4B(4, 6) - 2, e ~ r(4, 2) - 2 


Scheme 


n = 50 


n = 200 


n = 500 


Coverage 


Avg Length 


Coverage 


Avg Length 


Coverage 


Avg Length 


ECDF 


0.80 


0.49 


0.80 


0.09 


0.81 


0.04 


Smoothed 


0.93 


0.36 


0.95 


0.08 


0.96 


0.03 


FDR 


0.76 


0.30 


0.77 


0.06 


0.80 


0.02 




0.87 


0.43 


0.88 


0.10 


0.91 


0.03 


|-„9/i0i 


0.85 


0.46 


0.84 


0.09 


0.88 


0.03 


|-„14/15-| 


0.83 


0.48 


0.85 


0.09 


0.85 


0.03 



We can see from the table that the smoothed bootstrap scheme outperforms all 
the others in terms of coverage. It must also be noted that this is achieved without 
a relative increase in the lengths of the intervals. The m out of n bootstrap with 
[n^/^] also performs reasonably well. It clearly outperforms all other m out of n 
schemes as well as ECDF and FDR bootstrap procedures (which are inconsistent). 
Figure 1 shows the histograms of the distribution of Ti(Cn — Co) (obtained from 
1000 random samples) and its bootstrap estimates obtained from the 4 different 
bootstrap schemes (using 2000 bootstrap samples each) from a single data set of 
size n = 500 from model (1) with Z ~ 45(4, 6) - 2, e ~ r(4, 2) - 2, ao = -1, /3o = 
1, Co = 0. The histograms clearly show that the smoothed bootstrap (top right 
panel) provides, by far, the best approximation to both, the actual (top middle 
panel) and the limiting distributions (top left panel). In fact, the histograms of 
the distribution of n(C„ — Co) and the corresponding smoothed bootstrap estimate 
are almost indistinguishable. The m out of n approach, although guaranteed to 
converge, lacks the efficiency of the smoothed bootstrap. This may be due to the 
fact that we do not have an optimal way of choosing the tuning parameter m„. 
The smoothed bootstrap also requires the choice of a tuning parameter, namely, 
the smoothing bandwidth, but the in our analysis the results were very insensitive 
to the choice of the bandwidth. This is certainly an advantage for the smoothed 
bootstrap procedure. 



7 More general change-point regression models 

In this section we mention some of the broader implications of our analysis of (1) 
in the context of more general change-point models in regression. We can consider 
a model of the form 

Y = i;^,{W, Z)lz<Co + ^PoiW, Z)lz>Co + e, (25) 

where Z is a continuous random variable; is a random vector of covariates; 
ao G MP and Pq G M"^ are two unknown Euclidian parameters; ipa{w, z) and ^/^(w, z) 
are known real- valued functions continuous in {w, z) and twice continuously differ- 
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Figure 1: Histograms of the distribution of n{Cn — Co) and its bootstrap estimates: 
the asymptotic distribution of ra(C„ — Co) (top left); the actual distribution of 
n(Cn — Co) (top middle); the distribution of n(CjJ — Cn) for the smoothed (top 
right), ECDF (bottom middle) and FDR (bottom right) schemes; the distribution 
of m„(C* - Cn), mn = [ns] (bottom left). 

entiable in a and (3 respectively; Co ^ [o^, b] C supp(Z) C M is the change-point; e 
is a continuous random variable, independent of {W, Z) with zero expectation and 
finite variance > 0. We assume that ipaQiW^Z) is identifiable from ^p^^{W,Z) 
and that the least squares problems 

S 1 5Z (^^- - ^"(^^•' \ mm \ J2 - UW„ Z,)f \ 

U.<c J U.>c J 

are well-posed for every possible data set {(^i, Zi, Wi), . . . , (F„, Z„, Wn)} and any 
C G supp(Z)°. We also assume that ipaoiw, Co) 7^ ipoiw, Co) for every value of w. 
Like in the simple case, the method of least squares can be used to compute 
estimators a„, /3„ and C„. One simply takes the minimizer (a„, /3„, C„) of 

n 

i=i 

with the smallest C-component. 
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Since the simple model (1) is a particular case of (25), one can immediately con- 
clude from our analysis that the usual ECDF and residual bootstrap procedures 
will not be consistent. However, the smoothed bootstrap can be adapted to pro- 
duce consistent interval estimation. The modified scheme can be described as 
follows: 

1. Choose some procedure (e.g., kernel density estimation) to build a distribu- 
tion Fn with density /„ such that fn ^ f uniformly on some open interval 
containing (o w.p. 1, where / is the density of Z. Let and Pjf be the 
empirical measures of the centered residuals (as in the description of Scheme 
2 in Section 2) and Wi, . . . , Wn, respectively. 

2. Get i.i.d. replicates Z*^, . . . , Z* „ from F„ and sample, independently. 



4,1, • • • , <,n ■ K and W^^, ' Pjf . We could have also kept the 
l^.'s fixed, i.e., W^^ = Wi. 



3. Define Y^^ = ^.„(W^;„ ^;,)lz: xc„ + ^L^^lr Zl^^z^^^H. + 



j = l,...,n. 

4. Compute the bootstrap least squares estimators (a* , (*) by taking the 
minimizer of 



Although our analysis indicates that this smoothed bootstrap procedure must be 
consistent, it is difficult to use our methods to prove consistency in such generality. 
However, the proof of consistency for the simple model (1) can be adapted to cover 
the case of parametric additive models, i.e., when ipa{ui, z) and ip{w, z) are of the 
form 



with the smallest (^-component. 




p 





and 



k=l 



where gj,hk, j = I, ■ ■ ■ ,p, k = 1, . . . ,q are known smooth functions. 
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A Appendix 

In this appendix we provide the proofs of most of the results stated in the previous 
sections. We start with some results that characterize convergence in the space 
Vk with metric dx- 

A.l The space Vk 

Recall that is the space of all functions on C that are continuous in the 
first two co-ordinates and cadlag in the third. We keep the notation introduced in 
Section 3. 

Lemma A.l Let K = A x I cM."^ be a compact rectangle where A cM.^ and I C 
M. Let W a continuous function on K and {Wn)^=i a sequence of elements in 
such that dx {Wn, W) — )■ 0. Then, with the notation = sup {\x (h) \ : h ^ K}, 
we have that \\Wn — W\\j^ — )■ 0. 

Proof: Let e > 0. Since K is compact, W is uniformly continuous on K and 
therefore there is 5 > such that \W (9) — W{{})\ < | whenever 16* — < 6. Also, 
there is p > such that sup{|s — A(s)| : s E 1} < 6 whenever ||A|| < p. It suffices 
to choose P < ^ A ^ where L is the length of /. To see this, assume I = [a, b] and 
observe that for any r G (0, |), r < 2r — 4r^ < log(l + 2r) and for any r > — 1, 
log(l + r) < r. It follows that for any s e I, log(l - 2p) < -p < log < P < 
2p - 4p2 < log(l + 2p) and thus, |As - s| < 2{s - a)p < 2Lp. 
Now, since dx {Wn, W) — )■ 0, there is A G N such that for any n > N there exists 
A„ G A/ with the property that ||A„|| < p A | and 

sup {|W^„(a,/3,C)-lV(a,/3,A„C)|}<pA^. (26) 

(a,/3,C)ex ^ 

Then, for any 6 = {a,(3,() ^ K and any n > A we have that /3, A„,(C)) — 
(a,/3,C)| = |A„(C) - CI < ^ and thus, we can bound \Wni0) - W{9)\, by 

\Wr, (a, [3,0 -W (a, (3, A„(C)) | + \W {a, (3,0 - W {a, [3, A„(C)) | 
< e/2 + e/2 = e 
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using (26) and the uniform continuity of W. From this it follows that \\Wn — W\\ < 
e for any n > N. □ 

Lemma A.l shows that as long as the limit is continuous, convergence in the 
uniform and Skorohod topologies are equivalent. The next result concerns the 
continuity of the smallest argmax functional. 

Lemma A. 2 Let K G M.^ be a compact rectangle and W G Vk be a continu- 
ous function which has a unique maximizer 9* G K . Then, the smallest argmax 
functional is continuous at W (with respect to both, the uniform and Skorohod 
topologies) . 

Proof: Let {Wn)^=i be a sequence converging to W in the Skorohod topology. 
Let e > be given. Let G be the open ball of radius e around 6* and let 5 : = 
{W{e*) - sup0g^\G {W{e)}) /2 > 0. By Lemma A.l we have \\Wn - WW^ < 5 for 
all large n. If this condition is satisfied, then 

W{6*) = 25+ sup {W{9)}>5+ sup {Wn{e)} . 

eeK\G eeK\G 

But \\Wn - W\\j^ < 6 also implies that snp{Wn{0)} > W{e*) - 6. The combi- 

nation of these two facts shows that if \\Wn — W\\j^ < 6, then any maximizer of 
Wn must belong to G. Thus, |sargmax^g^{ 1^^(19)} — 9*\ < e for n large enough. □ 



A. 1.1 A convergence theorem for the smallest argmax functional 

Recall the definitions of S and Sj, where J C M is any interval containing 0, which 
were provided in Section 3. For a compact rectangle K = Ii x I2 x c M.^ 
containing the origin, consider the subspace of consisting of all functions 
ip G Vk which can be expressed as: 

00 

i/j (hi, h2, hs) = Vo{hi, h2)l + Vk{hi, ^2)lafe</i3<afc+i 

fc=l 

00 

(27) 

k=l 

where (. . . < a^k-i < ct-fc <...<ao = 0<...<afc< (^k+i < ■ ■ ■)keN ^ sequence 
of jumps and (Vfc)^,^^ is a collection of continuous functions. We write when K = 
M^. Observe that the representation in (27) is not unique. However, knowledge 
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of the function ip and of the jumps (a^) completely determines the continuous 
functions {yk)k&. Associate with every ip, expressed as in (27), a pure jump 
function ip E S whose sequence of jumps is exactly the a^'s, i.e., 

oo oo 

^{t) = J2^a,<t + J2la_,>t. (28) 
k=l k=l 

Finally, we denote by sargmax and largmax the smallest and largest argmax func- 
tional, respectively. 

The next lemma, which mimics Lemma 3.1 of Lan et al. (2009), makes a statement 
about the continuity of the smallest argmax functional on the space x Sj.^. 

Lemma A. 3 Let C E N, K = [— C, C]^ and ('il>n,'>Pn) , (V^o^'^o) be functions 

V / n=l 

in X i5[_c,c] such that ipn satisfies (27) for the sequence of jumps of ipn for 
any n > 0. Assume that {ipn,4'n) (V^cV'o) in I^k ^ S[-C',c] (with the product 
topology). Suppose, in addition, that ipQ can be expressed as (27) for the sequence 
of jumps (. . . < a_k-i < a-A: < • • • < ao = < . . . < flfc < flfe+i < . . .)fcgp^ ofipo and 
some strictly concave functions {Vj)j^z with the property that for any finite subset 
A G Tj there is only one j E A for which 

max <^ sup {Vm{hi, h2)}} = sup {Vj{hi, /ig)} . (29) 

Then, sargmax{ipn{h)} and largmax{iljn{h)} are well-defined for sufficiently large 

hi^K heK 

n eN and 

(i) sargmax{%ljn{h)} — t- sargmax{ipo{h)} as n ^ oo 
(a) largmax{ipn{h)} — )■ largmax{ipQ{h)} as n ^ oo. 

h&K h£K 

Proof: We can write ipn in the form (27) with (. . . < an-k-i < o-n-k < 

. . . < an,o = < . . . < a„^fc < a^M+i < • • OfceN being the sequence of jumps of V'n 

and Vnj being the continuous functions. Consequently, ipn, the pure jump process 

associated with ipn, can be expressed as (28) with jumps at {an,k)kei- 

Let Nr and Ni be the number of jumps of ipQ in [0, C] and [— C, 0) respectively. Let 

p > be sufficiently small such that all the points of the form aj ± p are continuity 

points of V'o, for —Ni < j < Nr. Since convergence in the Skorohod topology of 

ipn to ipo imphes point-wise convergence for continuity points of ipo (see page 121 

of Billingsley (1968)), and all of them are integer-valued functions, we see that 
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i'nidj ~ p) = j ^ ^ and i'nidj + p) = j for any 1 < j < A^r, and ipn{C) = for 
all sufficiently large n. Thus, for all but finitely many n's we have that ipn has 
exactly A'^,. jumps between and C and that the location of the j-th jump to the 
right of satisfies |a„j- — aj\ < p- The same happens to the left of zero: for all but 
finitely many n's, ipn has exactly A''; jumps in [— C, 0) and the sequences of jumps 
i^n-j)'i^=i, 1 < j < Ni, converge to the corresponding jumps a_j. 
Let V* = sup {Vj{hi, : h E K, —Ni < j < Nr}. Since all the V^'s are continuous 
and K is compact, this supremum is actually achieved at some value {h\,h*2) G 
[— C, C]^. By (29) and the strict concavity of the V^'s, it is seen that {h\,h2) 
is unique and that there is a unique "flat stretch" at which this supremum is 
attained. Suppose, without loss of generality, that the maximum value is achieved 
in an interval of the form [a^, a^+i A C) for a unique /c G {1, . . . , N^}. Now, write 
6o = 0; hj = ^^+^^^^+^ for 1 < J < Nr] and bj = for -A^^ < j < -1. 

Note that the b/s (for any value of the first two variables) are continuity points of 
both ipQ and ipo- 

Let K = min_Ar;<j<jv^,+i(C A aj — (— C) Vaj_i) be the length of the shortest stretch. 
Take < r], 5 < k/4. Considering the convergence of the jumps of ipn to those of 
ipo, there is G N such that for any n > N, the following two statements hold: 

(a) Consider p > such that if ||A|| < p, then 

sup{|s- A(s)| : s G [-C,C]} < 6, 

just as in the proof of Lemma A.l. By the convergence of ipn to V'o in the 
Skorohod topology, there exists A„ G A[_c^q such that ||A„|| < p and 

sup {\i:n{hi, h2, \n{h)) - 1po{hi, h2, h)]} < 7]. 
h&K 

(b) For any 1 < j < (respectively j = 0, — A"; < j < —1), bj lies somewhere 
inside the interval (a„j + 6,C A a„j+i — 6) (respectively (a„ _i + 6, a„,i — 6), 
((— C) V ctraj-i + S, anj — 6)). This follows from what was proven in the ffist 
two paragraphs of this proof. 

From (a) we see that |A„(6j) — bj\ < S for all —A"/ < j < Nr. But (b) and the 
size of S in turn imply that bj and Xn{bj) belong to the same "flat stretch" of ipn 
and thus ipnih, /i2, A„(6j)) = tpnih, /i2, bj) = Ynjih, /is) for all {hi, /ig) G [-C, C]^ 
and all — A"; < j < Nr. Considering again (b) and the second inequality in (a), we 
conclude that ||V^j — ^Hr_cc'i2 < V for all — A"; < j < Nr and all n > N. Hence, 
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all the sequences converge uniformly in [— C, C]^ to their corresponding 

Vj. Consequently: 

max < sup Vniihi^ho)} — > max < sup Vj(hi,h2] 

, J^^^^A^nAhi.hi)} — V max {■14(/ii,/i2)} = 14(/i*i,/i;), 
argmax {K,fc(/ii, /i2)} — > argmax {VkQii^hi)} = {hl^hl), 

lim max < sup VnAhi.ho)} < lim max IK, /lo)) • 

The above, together with (29) and the fact that an^k and an,k+i o,k+i, 

imply that 

saigmax{ipn{h)} — )■ {h\,h*2,ak) = sargmaxj^/'ol^)} 

largmax{V'„(/i)} {h\,h*2,ak+i) = largmax{?/'o(/i)} 
as n — > oo. □ 



A. 2 Some useful lemmas and proofs 

We first give an account of a series of technical lemmas which will aid us in the 
proof of Propositions 3.1, 3.2 and 3.3. 

Lemma A. 4 Let a 7^ /3 G M. Consider the class of functions from to M given 
by 

J2/ = {<j){y,z) := {y - al(_oo,C](-s) - P'^{(,oo]iz))li{z)\( eR,I CR is an interval] . 

Then, ^ is a VC-subgraph class with envelope \y\ + |a| + There is an upper 
bound for the VC-index of that is independent of a and (5. Moreover, there is 
a continuous, increasing function Jsi/ with 7^/(1) < 00, which is also independent 
of a and (3, and satisfies the following property: If 3l (Z is a subclass with 
envelope B and Wi, . . . , Wn is a random sample, defined on some probability space 
{Q,A,P), from a distribution fi for which ^{B'^) < 00 and /i„ is the empirical 
measure defined by the sample, then 

[ sup{|(/i„-/x)(^)|}t/P<^v/M^. 
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Proof: We use the same notation as in Lemmas 2.6.17 and 2.6.18, page 147 of 
Van der Vaart and Wellner (1996). Consider the classes of functions = {y — 

«l(-oo,c](^)-/31(C,oo]W :CeM}andJr= {l(_oo,c](^) : C e M}. Then, is a VC 
class with VC-index 2. It follows that M' = {13 — a) ■J(f+{y — /3) is also VC. Recall 
that = {li{z) : / C M is an interval}. Letting [ip > t] := {{y,z,t) : (p{y,z) > t} 
for (y9 G we see that 

t] : ^ e = X {J^ < 0} X (-00, 0)^ |J 

(^{[tP > t] : ij e n {R X {J^ > 0} X R)^ 

from which it follows that =2/ is VC. Moreover, the VC-indexes of and J-" are 
two and three for any choice of a and /3. Hence, the corresponding VC-indexes of 
Jif and =2/ both have upper bounds independent of a and /3. The existence of the 
function is a consequence of the maximal inequality 3.1 in Kim and Pollard 
(1990). Note that only depends on the VC-index of the class which in turn 
has an upper bound independent of a and (3. □ 



Lemma A. 5 Suppose that (I)-(IV) hold. Then, 
(t) \\Qn&z<i.)K.) - cr^nZ < (•) A Co)||[„,,] ^ 0, 

(tt) ||Q„(i^„iiz<(.)Acj - mmz < (•) A co)||[,,,] ^ 0, 

(Hi) \\<Q_n{ln'^Z<{■)^Cr.)\\ya,b] 

M ||Qn(lz<(.)AcJ-F(lz<(.)ACo)||[,,,] ^0. 

Also, these statements are true ifiz<{-)AC„ replaced by any o/l( )<z<Cn; lCn<^<(-) 
or lz>(.)vCn- 

Proof: Since — )■ C^q and Z is continuous, for any G [a, 6], we obtain 

|p(r2iz<CAcJ-P(>^'iz<CACo)| < P(r2|iz<a-iz<col)^o, 

|P(|r-«o|iz<CAcJ-P(l>^-"o|iz<CACo)l < P(|r||i^<c„-i^<^j)^o, 

|P(riz<CAcJ-P(riz<CACo)l < P(|r||iz<c„-iz<col)^o, 

|P(i^<CACn) - P(iz<CACo)l < P(|i^<c. - i^<col) ^ 0. 
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Also note that the convergence is uniform in G [a, b]. Thus, 



(>^'1Z<(-)ACJ-P(>^'1^<(-)AC0)||[„,,] 



< 



+ ||P(>^'1^<(-)ACJ -P(>^'l^<(-)ACo)||[.,,] ^ 

as n — 7- oo by (III). Similarly, we also obtain that ||Q„(|y — a;o|lz<(.)ACn) ~ 
P(|r-«o|lz<{.)ACo)llM ^ 0, ||Q„(yiz<(.)AcJ -n>^l^<(-)ACo)llM ^ O and 
||Qn(lz<(-)AC,J - IP(l2<(-)ACo)ll[a,b] 0. This provcs (iv). 

Finally, (z), (ii) and (iii) follow as consequence of the convergence a„ — j- and 
of the following inequalities: 

||Q„(6tlz<(.)AcJ - ^'nZ < (■) A Co)||[,,,] 
< ||Q„(F'lz<(.)AcJ - P(^'l^<(-)ACo)||[,,,] + 2|«n - aolQn(iri) + \al - all 
+ 2\ao\ ||Q„(Flz<(.)AcJ -P(>^lz<(.)ACo)||[,,6] + «o Qn(lz<(.)AcJ -P(l^<(-)ACo) 



[a,b] 



and 



||Qn(|e„|lz<(.)AcJ -P(|e|lz<(.)AcJ||[„,,] < 
,(|F-ao|lz<(.)ACn) -P(l^-"o|lz<(.)ACo)||[„,f,] + l"n-ao|- 



and 



||Qn(enlz<(.)AcJ||[,,,] < \\QniYlz<i.)AcJ-nYlz<i.)A(o)\\[aM 

+ \an - ao\ + \ao\ ||Q„(lz<(.)ACn) - P(l^<(-)ACo)||[a,b] • 
The other three cases follow from similar arguments. □ 



Lemma A. 6 Suppose that (I)-(IV) hold. Then, 
(t) ||(p;-Q„)(eaz<(.)AcJ||[„,,] ^0, 

Also, these statements are true ^/lz<(-)ACn replaced by any o/l(-)<z<c„, lcn<^<(-) 
or lz>(.)vG- 

Proof: By the maximal inequality 3.1 from Kim and Pollard (1990) and Lemma 
A. 4 we see that: 
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The lemma now follow directly as Q„(e^) — i- (a consequence of Lemma A. 5). 
The other statements are proven similarly. □ 

A. 2.1 Proof of Proposition 3.1 

Noting that in = Y - anlzKCn - Pn^zx,^, we write 

me{X) = -{la + an-aY'i-z<UAC-{^n + (3n-aY'i-u<z<( 

-{in + a„ - /3)^l^<z<c„ - (en + /?« - /3)^lz>c„vc> (30) 

and therefore 

-Kiil) = M*M <M:{ei) 

Letting 7* = (q;*,/3*), noticing that M*(^„) = -P* (e^), and by rearranging the 
terms in the above inequality, we get 

I7: - ln\^K{Z < a) A P;(Z > 6) < P: (etl.<z<6) 

+2|7: - ln\ {\K (enlz<a) | + (e„lz>b) |) • 

Consider P;(Z < a). By (u) of Lemma A.6 we see that |(P; - Qn)(^ < a)| 4 
and by {iv) of Lemma A. 5 we can show that |(Q„ — P)(Z < a)\ — )■ 0. Thus, 
combining the two, we have P*(2' < a) — )■ P(Z < a). Similarly, we can show 
that P;(Z < a) A P;(Z > 6) P(Z < a) A P(Z > 6) > and also that 
|P; (inlz<a) I + |P: (e„lz>fe) I 0. Also, observe that E (P:(e2)) = Q^iil) ^ a^, 
by assumptions (l)-(III) and so Pj^(e^) is bounded in L^. Hence, we can write 

i7:-7nr<op(i) + i7:-7niop(i) 

and therefore I7* — 7„| = Op(l) (and, consequently, I7* — 7o| = Op(l)). 
We first rewrite me(X) as follows: 

me{X) = -il - 2{an - a)e„lz<CACn - («« - «)^l^<CACn 
-2(/3„ - a)inlt:„<z<c - - ")^lcn<^<C 
-2(a„ - /3)inli:<z<Cn - («n - /3)^lc<z<Cn 
-2(/3„ - /3)e„lz>cvc„ - (/3n - /3)'lz>cvc„- (31) 
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We can then decompose M* as in (31), and use Lemmas A. 6 and A. 5 and the fact 
that On to obtain 

||M: + P:(et)-M„-Q„(et)||,, A 0. 
\\M: + K{^^)-M-a^\\^^ A 

for every compact K d Q. But 9q is also the unique maximizer of M + cr^ and 
\ln ~ 7o| = Op(l). Therefore, the conditions of Corollary 3.2.3 (ii), page 287 of 
Van der Vaart and Wellner (1996), hold and we obtain that 6'* — )■ 6q (and also 
that ^0). □ 

A. 2. 2 Proof of Proposition 3.2 

We will apply Theorem 3.4.1 of Van der Vaart and Wellner (1996) to prove the 
result. Let : X M3 ^ M be given by d{e, ^) = K^i, 62) - {A, ^2)! + V|^3 -^sl- 
Consider 77, p, L > as in (V) and a compact rectangle K C Q such that {9 E Q : 
d{6, 9n) < Tj for some n G N} C K. We can take L large enough so L > IV 
sup {\9i — d2\y \02 — "^il : E K}. Pick n large enough so we can fix some 5 G 
(^^,77). Then, taking also (I)-(IV) into account and possibly making rj smaller, 
we can find positive constants Ci, C2 > and A^ G N such that for any n > N, we 
have (5), (6), (7) and the inequalities: 

inf - A - > Ci, 

Q„(Z < a) A Q„(Z > 6) > C2. 

Also, let Mn{e) := M*{e) + P;(el) and Mni0) := M„(0) + Q„(e2) for all ^ G 6. 
Choose n > N and 9 E Q with | < d{9, On) < S. Then, considering the properties 
of the constants just defined and the expression 



MniO) - MniOn) = -2(a„ - a)Q„(e„lz<CAcJ - (ttn - a)'Q„(lz<CAC. 



- 2(/3„ - a)Qniinlu<z<c) - - a)'Q„(lc„<z<c) 

- 2(«„ - /3)Q„(e„lc<z<cJ - («n - /3)'Qn(lc<z<cJ 

- 2{Pn - /3)Qn(enlz>CVcJ ' " /3)'Qn(lz>CVcJ (32) 

it is seen that the sum of the 1st, 3rd, 5th, and 7th terms in (32) can be bounded 
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from above by While we also have, 

(on - a)^Q„(lz<CACn) > C2{an-af 
(/3„ - /3)'Qn(lz>cvcJ > C2(/3.-/3)2 



5 

2 



(/3n - «)'Qn(lc„<Z<c) > Cip|C-Cn|, if|C-Cn|>^> ^ 



K-/3)'Qn(lc<Z<Cn) > Cip|C-Cn|, if|C-Cn|>V> ^ 



and therefore, noting that either (a — a;„)^ + (/3 — > y or |C — Cn| > y, letting 
c = ]^C2 A (cip) and adding all the terms in the previous display, we get 

g r2 

sup {Mn{0) - MniOn)} < ' 2c6'^ Vu > N. 

^<d{e,e„)<5 \pnn 



Hence, setting 5„ = A we get that 



sup {Mn{Q) - -Mn(^n)} < "C^^ V 5„ < (5 < 7], > A^. (33) 

f«i(e,6i„)<5 
Next we will show 



{ 



v/^E I sup { I (M„ - Mn) {Q) - (M„ - A^„) (^n) I } I < ^<5. (34) 



yd(6»,e„)<<5 



Note that, using the expansion (31), M„(6'„) = A^„(6'„) = 0. To control the term 
(M„ — M.n){ff) observe that it admits a very similar expansion as (32) with the Q„ 
replaced by (P* — Q„); in particular, we can write the difference Miniff) — A^n(^) 
(by re-arranging the terms) as 

- 2(«„ - a)(P: - Q„)(e„lz<CAcJ " 2(/3„ - mi ' Q„)(e„lz>cvcJ 
-2(/3„ - a)(P; - Q„)(e„l^„<z<^) - 2(a„ - /3)(P; - Q„)(e„lc<z<cJ 

-(«„ - «)2(P; - Q„)(1Z<CACJ - (/3n - /3)'(P: - Qn)(lz>CVcJ 

-(a„ - /3)2(P: - Q„)(lc<z<cJ - (/3n - «)'(P: - Qn)(k„<z<c)- (35) 
Each of these terms can be controlled by using Lemma A. 4 as 



E(||(P:-Q„,)(e„lz<(.)AcJ||[.,,]j < -^VQn(6l; 



E(||(P;-Qn)(e„l(.)<z<cJ|||^_^,^l<,.) < ^7Qn(ellc„-.^<z<c„+.^ 
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Lemma A.5 implies that Qn{ellc^-s^<z<u+s^) ^ f^^P(Co - 5^ < Z < (o + S^) = 
a"^ {2 f {(q)6'^ +o{6^)} . Hence, there is a constant R> such that the right side of the 
above equations are bounded by R/y/m^ and -Ra/ 5^ + o{5'^) / y/mn- Using similar 
arguments, we can in fact make R large enough so that the following inequalities 
hold too 



E 



E 



n - Qn)(enlz>(-)VC„)||[a^5] 
^n)(enlc„<Z<(.))|||^_^„|<52 



< 



< 



R 



/nir, 
R 

frrTr, 



(36) 

V52 + o((52). (37) 



We also assume that R > Jr{l). Using (36), (37), the discussion preceding the 
display, and grouping two consecutive terms at a time in the expansion (35), it is 
easily seen that 



V^e( sup {\{Mr,-Mrm 

\ d(6i,e„)<<5 



-Mn)(^n)|} 



/nir. 



Thus by taking rj > Q small enough we can show that (34) holds for every n > N 
and any 6 G [5„, f]), with 5„ and N defined as in (33). Defining 0„(5) = ^5 and 
"^n = the hypotheses of Theorem 3.4.1 of Van der Vaart and Wellner (1996) 

are satisfied (note that Proposition 3.1 implies that d{9n,0^) — > 0). Therefore, 
rnd{9^, 91) = ^m^{(xl - a^f + m,(/3* - /3,)2 + v^m^lC* - Cn| = Op(l). □ 

A. 2. 3 Proof of Lemma 3.1 

Let > be an upper bound for the norm of the elements in K. The maximal 
inequality from Kim and Pollard (1990) and Lemma A. 4 imply 



/m^E 



^n)(enl/-.__u±i 



/m^E 



--Q„)(i,„,i^<,,,„ 



K 



K 



< J^(l)^Q„(ellc„_^<z<cJ 



By (z) and (zt>) of Lemma A.5 applied with lz<(.)ACn i'^ place of l(.)<z<Cn) we see 
that the righthand side of both the above inequalities go to zero. On the other 

K 

■ ■ ...... sn-r^^-.^^sn- ■ 

and ^/fn^\ 



hand, using (8) and (9) it is easy to see that both A/m„||Q„(e„l^ +-^<z<c 
(1^ ^ {.) ,^ )\\k converge to zero. Now, note that 
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and thus ^/m 



:-Qn)(e„l 



is bounded by 



K 





Qn (|en| 


K 





K 



K 



-^^ 0. Similarly we can bound 



from the expressions 



and show that it converges to zero in mean. Finally, 



Li 



-<Z<C,n 



we get that 



and 



ogous arguments, it is seen that 



C* - Cn 0. With completely anal- 

K 

Li „ , A L 



^ and 



D* — D 



^ as 



K 



well. Observing that En = An + Bn + Cn + Dn — Pn(e^) completes the proof of the 
result. □ 



A. 2. 4 Proof of Lemma 3.2 

It suffices to show that each of the components of {^n)'^=i is tight. Write inj = 
en{Zn,j,Yn,j) and let 

V TUn - 2m„ 

^ mn'^\'Qn\inf 

6 

Then, assumption (VIII) implies that r„ — )■ as n — )■ oo. Since the characteristic 
function of y^P* (e„l^<^„) is given by 

taking the limit as n — > oo we can conclude that ^m„F* (e„l2<^^) ^ A^(0,P(Z < 
Co)o"^) by using (VII) and the fact that (1 + /3„/n)" e'' if /3„ /3. With similar 
arguments, it is seen that ^m„P* (e„lz>(^„) A^(0,P(Z > Co)cr^)5 so the first two 
components of the random vector of interest are uniformly tight. 
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Consider now the processes r„(t) = m„P* (1^^^^<^^^_(^) and 

\l/„(t) = m„P* (e„l^^<^<^^_,_^). For any process \I/ G Vj, / C M compact interval, 

(5 > 0, we write 

wl (6) = sup{|^(ti) - ^(t)| A \^{t2) - 

where the supremum is taken over all ti < t < t2 E I with < t2 — ti < 6. Also, for 
any A G I, define (A) = sup {|^E'(t) — \E'(s)|}. This agrees with the notation 

defined in Chapter 14 of Billingsley (1968). Let > be an upper bound for the 
absolute values of the elements of J, consider any p > 0, and define the numbers 
and by. 



p neN 

Then, using Markov's inequality. 



lim P (sup{|^„(t)|} > ) < p (38) 
I5i P fsup{|r„(t)|} < p. (39) 



Now, let p, 7 > be any pair of positive numbers and assume that / = [a,b]. 
Then, choose 5 < ^ — tjtt^ A ^-^^ A -77^ so there is an integer N > 2 such 

' 8|fe-a|/(Co)^ 4 /(Co) ^ ^ — 

that 5 < i^^^ < 25. Define Sj = a + j^(b — a) and consider the partition 
{a = Sq < Si < . . . < sj^f = b} of J. Notice that if \E' is a step function on J, for 
w'^ (5) to be positive, we need at least two jumps in an interval of size at most 6. 
Then, the probability that at least two jumps of the process happens on any 
interval {sj-2, Sj] is bounded from above by 




mn{Zn,k - Cn),mn{Zn,l - Cn) ^ (Sj_2, Sj] 



2 



and hence the limit superior of the probability that either or r„ has two jumps 
in any interval of the form (sj_2, Sj\ is bounded from above by 2\b — a\^f{CoY/N'^ 
by (VI). Therefore, the probability that at least two jumps happen in any interval 
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of size at most S is asymptotically bounded from above by 

N N 

""^^rn^ < E - «r/(Co)VA^' < 4(iV - l)/(Co)> - a\6/N < 7. 

i=2 1=2 

Thus, 

mvV(wl^{5)> p) < 7 (40) 
The exact same argument can be used to show that 

P (w^^ {5) > p) < 7. (41) 

Now, note that 

y mn{Zn,j - Cn) G [a, + 5) > p 

< m„Q„ ( Cn + — < ^ < Cn + 

V "^n ran 
which implies that 

P {w^^ ([a, a + 5)) > p) < 5/(Co) < 7- (42) 

n— >-oo 

A similar analysis leads to the following bounds 

mvV{w^^{[h-6,h))> P) < 7 (43) 

n— ^oo 

P {wt„ ([a, a + 5)) > p) < 7 (44) 

n— ^00 

IS^ P («;r„ ([6-5, b)) > p) < 7. (45) 

n— ^00 

Putting together (38), (39), (40), (41), (42), (43), (44) and (45) and using Theorem 
15.3 of Billingsley (1968) we obtain that both sequences {^n)'^=i and (r„)^^ are 
uniformly tight in "D/. Similar arguments show the tightness of the third and fourth 
components of the process. Therefore, {^n)'^=i is uniformly tight. The uniform 
tightness of (-E*)^]^ now follows from the fact that {^n)^=i is uniformly tight and 
is a continuous function of S^. □ 

A. 2. 5 Proof of Lemma 3.3 

In view of Lemma 3.2, to show (i) it suffices to show convergence of the finite 
dimensional distributions. To this end, consider the real numbers t_Ar_ < . . . < 
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i_i < = ^0 < ^1 < ■ ■ ■ < tN+ and the linear combination 



j=i ^ V 

.7=1 ^ 



<Z<Cn + - 



^ + r/,m„P; 1 



Cn<2<C„- 



(46) 



where /x, A and the ^j's and the rj/s are arbitrary real numbers. Now, set — 
?7o = and define 



N± N± 

= ^V±k and X±j = ^^±k- 

k=j k=j 

Then grouping terms appropriately we can rewrite Wn as 



(47) 



Wn = /iV^Pn ^nl^^, *-iV_ 

-SSCnH 



mn — ^" mn 



+ 5^(A_jm„ + /Xv/^)Pi^ ( 

N. 



+ A^m^P; ( enl 



-<Z<CnH 



Using the independence of Xn,i) ■ ■ ■ ; Xn,mn-i characteristic function of Wn is 



.7=1 ^ 



3 

if-?. 



+Qn (e - 1)1 



+ 



e'v^'" - 1)1 



J=l 



m. 



5^Qn ( (e''^vfc+^^)^'"+^^'^^- - 1)1 



Cn+-^<^<Cn + — 



(48) 



Let r„ be given by 



e*^^" _ 1 _ i—^in + ^el 1, 



'■nir. 



Z<Cn- 



< 



s^Qn\el\ 



42 



Condition (VIII) now implies that r„ = o(l). But note that 



and so {%) of Lemma A. 5 together with condition (VII) and (8) imply that 



mnQn [ {e^'- - l)l^^^^^*-iv^ j = ^a¥(Z < Co) + o(l). 



Following a completely analogous argument one can show that 

rrinQn (^(e*^'" - l) ^ ^^^^^^^j^ = -^fx2p(Z > Co) + o(l). 
Now, take 1 < j < A^+, and observe that equation (8) implies 



nir. 



< \s\\y/m;,Qn |en|l. t^_, 0. 



Using (VI) we can write 

m„Q„ ( (e'^(7^+^^')^'"+'^^^- - 1)1^ 



= (^(.A,)e-^^-l)/(Co)(t,-Vi) + o(l) 
where is the characteristic function of e (under P). Thus, 

m„5^Qj (e-(^+^^)-+-'^^- - 1)1 ^ ) 

i=i 

Similarly, one can prove that 



AT. 



AT. 



J](t_,+i - t_,)/(Co)(^(sA_,)e-'^- - 1) + 0(1). 
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So putting (46), (47), (48), (49), (50), (51) and (52) together we see that. 



E{e''^") exp 



Lj=i 



k=j 



■^F(Z < Co) - '^nZ > Co) 



N+ 



3=1 



N+ 

k=j 



(53) 



But the right-hand side of (53) is precisely E (e*''^) where, with the notation of 
(11), W is given by 



k=l \ 0<i<i/i(-i_fc) 



t-k<0 



k=l \ 0<j<iy2{tk) 

and thus Wn W . From the fact that /x. A, the ^j's and the r^/s were arbitrarily 
chosen, by the Cramer- Wold device 



(S„(i-Ar_), . . . ,^„(t_l), . . . ,^„(tAr^)) (^(t_Ar_ ),..., ^(t_i), ^(ti), ^(^At^ )) . 



This gives the convergence of the finite dimensional distributions, proving {i). An 
application of the continuous mapping theorem shows that {i) implies {ii). Further, 
Lemma 3.1 and {ii) now imply {iii). □ 

A. 2. 6 Proof of Lemma 3.4 

Every sample path of E* = E*{hi, /12, h^) can be written as 

2h,Zi - h\¥{Z < Co) + 2/12Z2 - hlF{Z > Co) + lh,<o2{ao - /3o) Yl 

i=i 

-(ao - (3oYiyii-h3)lhs<o + lh3>o2(/3o - «o) ^ uj - lhs>o{ao - [iofv2{hz)- 

From this last expression it is obvious that for any fixed /13, the E*{-,-,h^) gets 
maximized at 0^ = Zi/P(Z < Co) and (f)^ = Z2/F{Z > Co)- The independence 
of the three co-ordinates follows from the fact that (p^ depends only on Zi, 02 
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depends only on Z2, and 03 depends only on u, v, Ui and 1^2. Since E* is piecewise 
constant in the third argument h^, to complete the proof it is enough to show that 
-E*(0^, 025 ^3) ~^ as {h^l —7- 00. But this follows from the law of the iterated 
logarithm (applied to the random walks defined by the fj's and Wj's) together with 
the fact that i^i{t) A i^2it) 00 as t —t- 00. Note that Yl'j=i^^^ of order 
0(-\/z/i log log z/i) a.s. as ^ 00. □ 

A. 2. 7 Proof of Proposition 3.3 

Lemma 3.4 and the fact that the Wj's and the fj's come from a continuous distri- 
bution, show that {E*, J*) satisfy the hypotheses of Lemma A.S, and in particular 
that (29) holds. Moreover, Proposition 3.2 shows that the sequence (^m„(a* — 
an), y/rfhi{l5:^ - /3„),m„(C* - Cn)) is tight. Now, consider C g N and let 0„, 0„,c 
and 0c be the smallest maximizers of En, En\[-c,c]^ and E*\]^_c,c]^- To prove the 
result, we will apply Lemma A. 3 and Lemma 3.3 of Lan et al. (2009). Using the 
notation of the latter, set e = ^, Wm = (pn,c, = 0c, Wn = 4>n and W = (j)*. 
From Proposition 3.2 we see that lim lim P {Wne 7^ Wn) = 0. Lemma 3.4 implies 
that limP (Wf^ 7^ W) = 0. Finally, Lemma A. 3 and an application of Skorohod's 

e-S>0 

Representation Theorem (see Theorem 1.8, page 102 of Ethier and Kurtz (2005)) 
show that Wne We and hence, from Lemma 3.3 of Lan et al. (2009), we conclude 
that (pn 4>*- □ 

A. 2. 8 Proof of Lemma 4.1 

We expand mg{X) as in (30) but with e = Y — aolz<(o ~ (^o^z>Co place of e„ to 
get 

mg{X) = -(e + ao - a)^lz<CoAC - (e + /3o - tt)^lCo<^<C 

-(e + «o - /3)'lc<^<Co - (e + /3o - /3)'lz>Covc- (54) 

Letting 7^ = (a„,/3„), we can also bound M„(^o) using a similar argument as in 
the proof of Proposition 3.1 to obtain 

\% - lo\^Fn{Z < a) AFn{Z > b) 
< P„ {eHa<z<b) + 2|7„ - 7o| (|F„ (elz<a) | + |IPn (elz>fe) I) • 
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By the strong law of large numbers 

P„(Z < a) A P„(Z > b) 

Therefore, w.p. 1 we can write 

1% - 70^ < 0(1) + |7„ - 7o|o(l) 

and thus the sequence (7„ — 7o)^i is bounded w.p. 1. 

Now, take any compact set C O and consider the classes of functions 

/Ci = {(e + ao-a)^l{-oo,CACo]}eex 

/C2 = {(e + /3o -tt)^l(Co,C]}ee/^ 

/C3 = {(e + ao-/3)^l{c,Co]}eGi^ 

/C4 = {(e + /3o -/3)^l(cvCo,oo)}ggJ^• 
If A* is an upper bound for the norm of the elements in K, we can see that each of 
these classes is a VC-subgraph class with integrable envelope (|e|+A* + |7o|)^. With 
the notation \\Q\\jr = sup {\Qf\ : / G J-"} for classes of functions J-" and probability 
measures Q, a combination of Theorems 2.6.7 and 2.4.3 of Van der Vaart and 
Wellner (1996) shows that all four quantities ||P„ — P||^_^, j = 1,2,3,4, converge 
to zero almost surely. Therefore using (54), we get the inequality 

l<j<4 

which now implies (i) ( Since Mn,M G T>x, ||M„ — M||^ is measurable.). The 
second assertion follows immediately from {ii). 

Consider a family of compact rectangles 9„ C 6^+1 such that 6 = U^^^G^. Then, 
since the sequence (% — 7o)^i is almost surely bounded, w.p. 1 we have that there 
is some m E N such that contains both 9q and the entire sequence (^n)^i- 
Finally, from (31) with 6n replaced by 6q it is seen that 

M{e) = -a' - (ao - «)'F(Z < C A Co) - («o - /3)'F(C < ^ < Co) 
-{a - /3o)'F(Co <Z<C)-Wo- P?nZ > C V Co)- 

As ao 7^ (3o and Z has a strictly positive density on [a, b], the last equation shows 
that M satisfies the conditions of Lemma A. 2. Since the event that M„ — )■ M 



F{Z < a)A¥{Z > b) 
(t¥ {a<Z<b) and 
0. 
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in T>Q^ for all A; G N has probability one, Lemma A. 2 allows us to conclude that 
sargmax(M„) = 9^ 9q. □ 



A. 2. 9 Proof of Lemma 4.2 



Let p,6 > 0. We know from Corollary 3.1 that the sequences {y/n{an — ao))^i, 
n{Cn - Co) and (raP„ (Co - ^ < ^ < Co + for any h > 0, are all stochas- 

/ n=l 

tically bounded. Thus, since m„ = 0{n) there is L > such that P (^ninlCn — Col > ) < 
p and P [^ymn\c^n — «o| > < P for any n G N. Therefore, 



P ( ml 



Pn(Cn + — < ^ < Cn 



> 6 



K 



< ( P„ (Co - — <Z<Co + —))+P (m„|Cn - Col > L 

< /(Co)^^mr + o(mri)+p, 

so by letting n — )■ oo and then p — )• we get [i). 

We prove (ii) for when p = 1, the case p = 2 follows from similar arguments. Note 
that if m^lCn — Col < L, then m^||P„(|e„|l^ , iO_^v<; )||_ft: can be bounded by 



^I^Co-^ + ^<2<CoH 



K 



+ ml\an - ao\ 



\{Co- — + — <z<Co + — 

nin nin nin 



But just as in the proof of (i), we have 



K 



< p to: 



< 



(I 



ell 



5 

K>2 



P (mZ\an - ao|P„(Co - — + — <^<Co + — )>| 
\ TO„ m„ TO„ 2 

2toL 



(Pn(l 



ell 



^ + 



L 1 L 3 

P ( TO7Jd„ - ao|Pn(Co + —<Z<Co + )>:^ 

TO„ m„ TO„ 2 



< /(Co)E(|e|) 



2(^ + 2£) . 



O TO ' + 



P ( TO;^|a„ - ao|P«(Co - — + — <Z<Co + — )>!^ 

TO„ m„ TO„ 2 



("T-nlCn - Col 



> L 



P TO„|C„ - Col > i 



The result follows again by letting n — )■ oo and p — > 0. □ 
The next results will be useful to support our conjecture of inconsistency of some 
of our bootstrap scenarios. 
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Lemma A. 7 Let X,B > 0, p E (0, |) and Hx he the distribution function of 
a Poisson random variable with mean A. For each value of A write L''^j^g = 
min{n G N : Hx^sin) > p} and = max{?2 G N : 1 — Hx{n) > p}. Then, there 
zs A* > such that L*^^^ < for all A > A*. 

Proof: Let cx be the median (i.e. cx = min{n G N : Hx{n) > ^}.) of Hx- Observe 
that Cx < U^. According to Hazma (1995), \cx — A| < log(2) for any positive A. 
Letting [a;J denote the greatest integer less than or equal to x, we have 

\Hx+Bicx+B) — Hx+Bicx)\ 

< \HxM^ + B + log(2)) - HxM^ - log(2))| 

< (i? + 21og(2))e-(^+^)i^±|^^^0asA->oo. 

as the Poisson mass function has a maximum at [A + 5J . Therefore, 

hmx^oo H\+b{Ux) — V2- But we also note that sup„gj^{i/A+B(^+l) —Hx+sin)} — )■ 
as A — 7- oo. Thus, 

JhR HxML'x+B + 1)=P<J< lim HxMU'x)- 
It follows that > L^x+B sufficiently large. □ 



Lemma A. 8 

Let X, B > 0, < p < ^, fi and v be two nondegenerate Borel probability 
measures on M and H^ x denote the compound Poisson distribution with inten- 
sity A and compounding distribution fi. For each value of A write L'^^^^^ = 
inf {seR: H^,x+b{s) > p} and f/^^ — sup {s G M : 1 — H^^x{s) > p}- In addition, 
assume that J x'^i'{dx),J x'^fi{dx) < oo and that J xv{dx) < J xfi{dx). Then there 
zs A* > such that i^^^^+B < ^^ix f^"^ '^^^ A > A*. Moreover, let < r < 1, suppose 
that there is another Borel probability measure 7 on M and define := )^^7 + 
^^x+B^ ^''^^ corresponding constant L^^ = inf |s G M : Hy^^x+B{.s) > p}. 
Then there zs A* > such that L^^ < a /^'^ '^^^ A > A* . 

Proof: Denote by $ the standard normal distribution and Zq the lower a-quantile 
of $ (i.e. $(za) = a). Also, write := / xfi{dx), d^ := J x'^fi{dx) and define the 
corresponding quantities c,, and d^, for u. For any possible value of A and p denote 
by T^,A a random variable with distribution i/^,A- It is easily seen (as, for instance. 
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T A 

in Theorem 2.1 of M5hle (2005)) that Sf,x ■= ^'^ ~ $ as A ^ oo. Since 

the standard normal distribution is continuous, the distributions of S'^^a converge 
uniformly on M to $ as A — oo. 

Let 1 < K < l/(2p). Then, since the distributions of S^^x converge uniformly to 
there is Ai such that 1 — $ I — ^ ) < /tp for A > Ai and A2 > such that 



$ "'^'^f = — < up for all A > A2. These two inequalities in turn imply that 

K,x+B < (A + B)c, + ^{X + B)d,z,p. 
Since c,, > Cu we can find A^ such that 



(A + B)cu + a/(A + B)duZ^p < Ac^ - ^JXd^z^p for all A > A3. 

The first part of the result now follows by taking A* := A1VA2 VA3. To prove the re- 
sult for the measure v-^ it suffices to see that we also have ( ^ ^1 ^ 

" V(A + 5)4^ 

as A — )■ 00 (this is easily seen by analyzing the characteristic functions). The 
rest follows from the same argument used to prove the first part of the lemma. □ 



A. 2. 10 Proof of Lemma 4.4 

Proof of (z): Let s < t. Note that {Zn)'^=i is a collection of i.i.d. random variables 
and nP„(^o+^ < Z < Co+^) is permutation invariant, so the Hewitt-Savage 0-1 law 
(see page 304 of Billingsley (1986)) implies that any convergent subsequence must 
converge to a constant. On the other hand. Lemma 3.3 implies that nFn{Co + ^ < 
Z<(o + i)^ Poisson((t - s)/(Co)). Therefore, (nP„(Co + ^ < ^ < Co + 
has no almost surely convergent subsequence. 

Proof of (u): Now, let 6 G (0, |). From Proposition 3.2 we know that there is 
Bs > such that P ^n|Cn - Col < Bs^ > 1 - 5 for any n G N. Choose h > 2Bs 
and take any increasing sequence of natural numbers n^. Write = Uk^nkiCnk < 
Z<Cn, + 4), Sk = nfcP„,(Co - f < Z < Co + ^) and T, = nfcP„,(Co + f < 
^ < Co + ^)- Then, |nfc|C„, - Col < Bs} C > f ^ > T^} and therefore we 
have P (fk > Tfc) A P (^Sk > T^) > 1 - 5 for all k. 

We know that Tk ^ Poisson((/i - 2Bs)f{Co)) and Sk Poisson((/i + 2Bs)f{Co)), 
so in view of Lemma A. 7 with B = 4i?5/(Co) and X = {h — 2Bs)f[(o), there 
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is a number > 2Bs large enough so that whenever h > we can find two 
numbers Ni^h < ^2,/i ^ N with the property that, hm^^^ P (T^ > N2^h) > 25 
and linifc^^ P (^fc < A^i,,,) > 26. Thus, for h > K, P {Tk > N2,h) > 26 and 
P {Sk < ^i,h) > 25 for all but a finite number of /c's. Therefore, for any k large 
enough, P (T^ > A^'a,/,) AP {Sk < Ni^h) > 26. Using the fact that P (^Sk >fk> > 

1-6 we get that P (fk > Tk > N2,h^ A P (^Ni^h >Sk>fk^>6 for all but finitely 
many k's. Thus, whenever h> h^,, 

P (fk >Tk> N2,h,^.o.) > 5 and P (^N^^h >Sk> fk,t.o.') > 6. 

But for every k E N, the events |Ta; > > A^2,/i| and ^Ni^^ > Sk > Tk^ are 
permutation-invariant on the i.i.d. random vectors Xi, . . . , Hence, the Hewitt- 
Savage 0-1 law implies that P ^T^ >Tk> N2,h, i-o.^ = 1 and P (^Ni^h ^ Sk > Tk, i.o 
1. Since Ni^^ < A^2,/i it follows that Tk = nfcP„^(C„^, < Z < + h/uk) does not 
have an almost sure limit. But the choice of the subsequence Uk was arbitrary 
and independent of K so we can conclude that for any h > h^,, the sequence 
< n¥n(Cn < ^ < Cn + ^) f docs uot couvergc in probability. Proceeding analo- 

l J n=l 

gously, we can prove the same for < nP„(Cn — ^ < ^ < Cn) r 

I J n=l 

Proof of {iii): We introduce some notation, for any two Borel probability mea- 
sures /i and z/ on M we write for their convolution and for A > we write 
CPoisson(/i, A) for the compound Poisson distribution with intensity A and com- 
pounding distribution yU. Let /Xq, and be, respectively, the distributions under 

P of 0(e + ao) and 0(e + /3o). 

Observe that depending on whether t<0,s<0<tors>Owe have that 
inFn{(l){Y)l^^^^i^^z<CQ+l) converges weakly to CPoisson(/i„, (t - s)/(Co)), 
CPoisson(/i„,s/(Co))*CPoisson(/i^,t/(Co)) or CPoisson(/i^, (t - s)/(Co)), respec- 
tively. This follows easily from convergence of the corresponding characteristic 
functions. Considering that {{Yn, Zn)}^=i is a collection of i.i.d. random vectors 
and that nP„(0(F)l^^_,_s ^^<^j^^a) is permutation invariant for (Yi, Zi), . . . , (1^, Z„) 
the same argument as in (i) applies here as well. 

Proof of (iv): We keep the notation used in the proof of {Hi). The argument here 
is quite similar to the one used to show {ii). Assume without loss of generality 
that < 0. 

Now, let 6 E (O, ^) and N E N. From Proposition 3.2 we know that there 
is Bs > such that P (^n\^n — Col < Bs^ > 1 — 5 for any n E N. Choose 
h > 2Bs and take any increasing sequence of natural numbers n^. Write T^^ = 



50 



nfcP„^(0(F)l^^ j^), = rikKMY)! B, and 

= r.,P„J0(y)l^^^^^^<^^^^). Then, {nfc|C„, - Col < Bs] C {sf,, < f,^, < T,^,} 
and therefore we have P {fl^^ < T^j^^ A P (^Sf f^ < f^^^ > 1 - 5 for all k. 
We know that T^^ -w CPoisson(/i/3, {h — 2Bs)f{Co)) and 



-w CPoisson(/i„, 255/(Co))*CPoisson(/i^, (/i + Bs)f{Co)) 

(B h -\- B \ 

as — )■ cxD. 

An application of Lemma A. 8 with /i = z/ = /i/?, 7 = /Iq,, _B = 4:Bsf{(o), t = | 
and A = (/;, — 2Bs)f{(o), shows the existence of an > 2Bs large enough so that 
whenever h > h^: we can find two numbers Ri^h > R2,h ^ N with the property that 
lim^_,^ P (r,^^ < i?2,h,) > 25 and lim;,.,^ P (^J^ > Ri^t)j > 26. Thus, for h>K, 

P (t^^ < i?2,h) > 26 and P > Ri,h^ > 26 for all but a finite number of A;'s. 

Therefore, for any k large enough, P ^T^/^ < R2,hj A P ^5^;^ > Ri,h^ > 26. Using 

the fact that P (^f,, < f,*, < T,*,) > 1 - 5 we get that P (fl, < T^^, < R,,^) A 

P (yRi,h ^ < > 6 for all but finitely many fc's. Thus, whenever h > h^:, 

P (rih < Tl, < R,,n.i.o) > 5 and P < < fl,,i.o) > 6. 

The argument relying on the Hewitt-Savage 0-1 law applied in the proof of (ii) can 
be used to finish this proof. 

A completely analogous proof apphes for < nP„(0(F)l? _zi<7<; ) r • '-' 
A. 2. 11 Proof of Lemma 4.6 

We start by computing the characteristic functions of the weak limits of the last 
two components of the process En as defined in (17). Let g-niO i'niO be the 
(unconditional) characteristic functions of nP* (1^^^2'<Co+-) ^^n(^lco<^<Co+-)' 
respectively. Fix ^ G M and write 



A, := (e'^"''"^^'co<-<Co+|) 
*n := nP„((e^«^-l)l^„<^<^„+, 

l<k<u{t) 
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where {i^{s))s>o is a Poisson process with rate /(Co) independent of (e„)J^]^. Then, 
i^niO = E (A„) and |A„| < 1. By the conditional independence of the bootstrap 
samples, we have 

An = ( 1+ -^n 

y n 

We now consider the characteristic functions of the complex-valued random vari- 
ables Taking into account the independence of the X's, we obtain that for any 
T) e 

E ^g*'7iRe(*„)+j7;2lm(*„)^ _^ ^t/(fo)E(e"5l(™=(?^)-i)+'''2 ^ ^gi»?iRe<I'J+ir)2lm<I'J j 

Therefore, \E'„ and, from the continuous mapping theorem, A„ e*«. Thus, 

Lebesgue's Dominated Convergence Theorem implies 

MO = E (A„) ^ E (e*?) = e'^(<o){E(o^"'-^yi) V ^ e M. (55) 
With simpler arguments, we can also show that 

^?n(0^e*^^^°^^'''''""~'^ V^gM. (56) 

While (56) is immediately recognized as the characteristic function of a compound 
Poisson process with rate /(Co) and compounding distribution Poisson(l), the char- 
acteristic function in (55) can be shown to correspond to another compound Pois- 
son process which can be written as 

l<j<u(t) 

where (t„)5^^ Poisson(l), (z/(s))s>o is a Poisson process with rate /(Co); and 
(T"n)^i, (eri.)J^i and {i^{s))s>o are mutually independent. 

Therefore, the fifth and sixth components of S„ as defined in (17) converge, re- 
spectively, to a compound Poisson process with rate /(Co) and Poisson(l) as com- 
pounding distribution and to the process described in (57). A similar analysis 
shows the analogous results for the third and fourth components of H„. The first 
and second components of can easily be seen (by using the Lindeberg-Feller 
Central Limit Theorem) to be asymptotically normal with mean and variances 
(j2p(Z < Co) and a'^F{Z > Co), respectively. 
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All these facts indicate that the finite dimensional distributions of the limiting 
process of H„ match those of the process H. In fact, we can proceed as in the proof 
of Proposition 3.3 (i.e., proving tightness and convergence of the finite dimensional 
distributions using the Cramer- Wold device) to show (i) and (ii). For the sake of 
brevity, we omit the full technical details. 

Then, arguing as in Proposition 3.2 one can show that the sequence {^/n{a'^ — 
ao), y/n{f3^ — f3Q),n{(* — ^o)) is stochastically bounded and then conclude that the 
(unconditional) asymptotic distribution of {y/n{a^ — ao), \/n{f3^ — f3o),n{(* — (q) 
is that of sargmax;jg]g3{-E'*(/i)}, with E*{h) as defined in (18) and (19). For the 
sake of brevity we omit the full technical details of these arguments. 
As n(Q — (o) = n((* — (n) + n'{Cn — Co), and if the ECDF bootstrap were consistent, 
the conditional distribution of n{(* — („) (given the data) and the unconditional 
distribution of n{(n — Co) would have had the same weak limit. Then, as a conse- 
quence of Lemma 3.1 in Sen et al. (2010) (also see Theorem 2.2 in Kosorok (2008a)) 
the unconditional asymptotic distribution of n{(* — Co) must be that of the sum of 
two independent copies of the asymptotic distribution of the n{zetan — Co)- The 
result now follows. □ 

A. 2. 12 Proof of Lemma 4.7 

Let Gn be the ECDF of ei, . . . , e„. We first observe that 
and hence, for any G M with |,^| < rj we have. 



< |r/|P„(|l„-e|) 

but P„ (|e„ — e|) is bounded from above by 

Ittn - ttol + (|ao| + |/3o|) |IPn(lz<C„ - l^<Co)l + \(^n " Po\ 

which goes to zero almost surely as consequence of Lemmas 4.1 and A. 5 (if), with 



tn n 



P„. Thus, 



sup 



e-*„(.)-e-/e-*„, 



X] 



a.s. 







and {i) follows immediately because e„ = Pn(en) -—^ and G„ converges to G in 
total variation distance with probability one. The second assertion is seen to be 
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true at once because G is assumed to be continuous and condition {%) implies that 
the characteristic functions of converge to the characteristic function of G on the 
entire real line with probability one. Statements {ii) and {Hi) are straightforward: 
On the one hand, we have shown that conditions (I)-(IV) hold for the ECDF, so 

Lemma A.5 implies that J x^dFl^{x) = P„(e^) - Pn(e„)^ ^ a\ On the other 
hand, 



\x\dFl 



lelc/P. 



= |P„(|l„-e„| - |e|)| 
< P„(|e„-e|) + |e„| - 



0. 



To prove (iv), we first notice that 

\x\^dFl^{x) < \en\^ + 3|e„|2p„ + 3|e„|P„ {il) + P„ . 



I 



Then, from Lemma A. 6 all but the last summand on the right-hand side converge 
almost surely. Hence, it suffices to show that limPn(|e„p) < oo w. p. 1. With 
this in mind, let L„ = |ao| + \an\ + |/3o| + \f^n\ and observe that 

Pn (lenH < Pn (|e|') + 3P„ {\e\^) L„ + 3P„ (|e|) Ll + 

The result then is an immediate consequence of the third moment assumption on 
e, the strong law of large numbers and the almost sure convergence of the least 
squares estimators. □ 

A. 2. 13 Proof of Proposition 4.2 

Just as in the proof of Proposition 3.1 we have 



Rn{On) 



k=l 



1 

n ^ — ^ 



from which we can see that 



|7:-7„|2p„(Z<a) AP„(Z>6) < 



n ^ — ^ n 



7ri 



+ 
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But the first of tlie terms on tlie riglit-liand side of the previous inequahty is 
conditionally bounded in Li (an upper bound for the conditional expectations is 

sup<^ / x^ciP;;(x) } < oo). The terms - V'e* .lz.<a and - Y^e* .lz,>fe both 
neN U J 

have zero conditional expectation and conditional variances equal to — P„(Z < 

n 

a) J x^(iP^(x) and —Fn{Z ^ ^) J x'^dF'ni^) respectively. So we have that 

^ n 1 " 



Thus, 



I7* — 7„| = Op^.(l) almost surely. (58) 

Now, let be the k-th order statistic from the sample {Zi, . . . , Zn) and a 
number such that = Z,.^. For any ( G [a, 6] define = max{l < j < n : 
Z(j) < ( /\Cn} and observe that we have 



1 " . 1 

j=l l<i<m.<; 



(59) 



and thus 



sup 



n 

-E 

n ^ 



< max 

l<fc<n I n 



(60) 



But the indexes and the order statistics are functions of Zi, . . . , Z„ and there- 
fore X-measurable. Hence, conditionally, e*^^ l^^ <cac„ ^ square integrable 



martingale with zero expectation. Hence, from Doob's submartingale inequality 
(see Wilhams (1991), Theorem 14.6, page 137) we get 



Px max 

\<k<n I n 



>P\<\ 



Pn(et) 



and consequently, equations (59) and (60) show that 



„ 2^^n,j'^Z,<(■)^U 



> p 



< 



[a,b] 



Pn(et) 



(61) 
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Similar arguments give that (61) is also true if we replace by any of 

l(-)<^.<Cn' lcn<z,<(-) lz,>(.)vc„- if ^"^^ li^^ 

2 " 

i=i 

2 " 

2 " 

i=i 

-^(/3n - /3) ~<Az>CVU - (/^n - /3)'Pn(lz>CVcJ' (62) 

(m) follows immediately from (61), applied for all the four possible types of indi- 
cator functions. Note that the four terms on the far right of all the rows in the 
previous display vanish when we subtract M„ from Lemma 4.1 shows that 
(a) implies [i), while Corollary 3.2.3 {ii), page 287, of Van der Vaart and Wellner 
(1996) together with (58) allows one to derive (iii) from (i) and (ii). □ 

A. 2. 14 Proof of Lemma 4.8 

The proof is analogous to the proof of Lemma 4.5. We again consider the number 
> defined in the statement of Lemma 4.4 and take C to be any compact 
rectangle containing the point (0, 0, /i*). To prove the theorem it suffices to show 
that the sequence (-E„(0, 0, /^a)),^! does not have a weak limit in probability when- 
ever /13 > and (0, 0, h^) G K. But in view of Lemma 4.4 this is straightforward 
because the (conditional) characteristic function of -E„(0, 0, h^) is given by 



Pni(n<Z<(n+'f) 



and Lemma 4.7 and the strong consistency of the least squares estimator imply 
that 

Thus, for all in a neighborhood of the origin, this characteristic function will 
converge if and only if nP„(C„ < Z < (n + ^) converges. We know that this is not 
the case from Lemma 4.4. □ 
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A. 2. 15 Proof of Proposition 5.1 

We will show that conditions (I)-(V) in Section 3 hold w.p. 1 for the bootstrap 
measures arising in this scheme. Note that (IV) is a consequence of Lemma 4.1. 
That ||Q„ — P||jr — — follows immediately from the fact that ||-Fn — -^||oo 0. 
Now, for any g = yip & Q with ip E J^, we have 

F{g) = aoP(lz<CoV^) + /3oP(lz>CoV'), 
from which we see that 



/3n ^ /3o 



+ (|«o| + |/5o|) 



+ (l«o| + |/3o|) / |1,<; -l.<Col/n(^)rf^. 



Lebesgue's dominated convergence theorem shows that the last integral goes al- 
most surely to zero and the strong consistency of the least squares estimators and 
property (I) now yields ||Q„ — P||g — )■ 0. Finally, we can write any h G H in the 
form h = y'^ip for some ip E J^. Using this representation we obtain, 

F{h) = alF{lz<Coip) + P'on'^z>Co^) + cr'F{,p), 
and the triangle inequality then implies that 

IIQn - P|k < - «ol + - /3ol) + («0 + + WQn " P||^ 

+ |P^(et) - F{e')\ + {al + ^1) [ |1,<^„ - U<M^)d^ ^ 0. 

It remains to show (V). Observe that (6) and (7) hold automatically because under 
Q„, in and Z are independent. Hence, we only require to show that (5) holds w.p. 
1. As (23) holds, we have 

inf l/„(C)}^ inf |/(C)}>0. 

The mean value theorem implies that for any Cj'C ^ [c, c?], there is G [0, 1] such 
that |F„(C) - FniOl = CI/n(C + ^(^ - C))- It follows that for > small 
enough, 

inf |-^|F„(C)-F„(Ull > inf lA(C)| VnGN 
and consequently (V) holds w.p.l for all S < rj for all large n. □ 
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A. 2. 16 Proof of Proposition 5.2 

We already know that conditions (I)-(V) liold w.p. 1. Condition (VII) holds 
automatically because Z and e„ are independent under Q„ and Qn(en) = 0. Lemma 
4.7 (f) implies that condition (VIII) holds a.s. It remains to prove (VI). 
Write / = [c, and consider the sequence of events {v47v}^gpj given by 

b " T] 

Xn -\ — & I, almost always, V 5, ?7 G (0, A^) 



A 



N 



Cn 



n 



ll/n-/||/->0 



n n 

Fix G N, let ip be the function iIj{x) = e*^^ for some ,^ G M or the function 
ip{x) = \x\^, p = 1,2, and r], 5 > be any positive real numbers smaller than A^. 
Then, 

Lemma 4.7 implies that (■?/') P {'ip{e)). And, when Aj\f holds, we also have 



n / Ux)dx - \ f{x)dx <2N U-f ^0. 

Hence, condition (VI) holds for all < 5, ?7 < A on A/^r. But the strong consistency 
of the least squares estimators and the conditions on /„ imply that each of these 
events have probability one. Therefore, P (flAreN^Af) = 1- Hence, condition (VI) 
holds w.p.l and the result follows from an application of Proposition 3.3. □ 

A. 2. 17 Proof of Proposition 5.3 

Since Q„ is just the ECDF, the validity of conditions (I)-(IV) follows from the 
result established for the regular ECDF bootstrap and Lemma 4.1. (VIII) is a 
consequence of the strong law of large numbers. It remains to show (V)-(VII). 
We start with (VI). First observe that m„P(^(e)l. _^^z<c +^) ('^+^)/(Co)P(V'(e))- 
We will proceed as follows: we will first use this simple observation just made to 
show that the following equations are true. 





rrin 


Pn(V^(e)l,„_±I<^<,,) - ( 


)P(^(e))/(Co) 


K 


P ^ 





(63) 




rrin 




)P(^(6))/(Co) 


K 


P ^ 





(64) 


rrin 


Pn(V'(e 




)^Co-ii<z<c„) 


K 


P ^ 





(65) 


Uln 


Pn(^(e 






K 


P ^ 





(66) 
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for any compact interval C M. All these facts put together will give 



m,. 



rrir 



C.n<Z<U + 



n) - (•)p(^(e))/(Co; 



K 



K 







(67) 
(68) 



for any compact interval K (ZM.. Having achieved this, we will be able to conclude 
that (VI) holds in probability. For if (67) and (68) are both true, we can take 
an increasing sequence of compacts {Kn)^^i whose union is M and then for any 
subsequence (n^)^]^ find a further subsequence (n^J^^^ such that 



P m„ 



)-(.)P(V(e))/(Co) 
J - (•)P(VXe))/(Co) 











> 



















> 











The Borel-Cantelli Lemma will then imply that (VI) holds almost surely for the 
subsequence (n^J^^. Therefore, it suffices to show (63), (64), (65) and (66). 



First consider the case where 
and write 



and a positive number rj > 0. Let t G 



rrir 



-re 



n 



Ho<z<Co+^ 



Then, IrJ < t'^a^^m„F(l 



n'ry^Co<Z<(o + - 



— i- 0. The characteristic function of 



m„P„(|e|lA,<2< I j_) can be written as 

^nit) = (l + tP(H)P(lco<z<Co+^) + -Y ^ e^^-^d^D/^^o) 



and therefore 



But 



m„P„(|e|l^„<z<Co+;^) r//(Co)P(|e|). 



sup <J E ( m„ 



< oo 



and hence the sequence of processes (mnPn(|e|lA ^z<c +-^) ) tight in r'[o,jy]. 
It follows that 

mr,FM\,^z<Co+^) (■)/(Co)F(|e|) in Pp,,] 
but since the limiting process is continuous and deterministic we actually obtain 



And with similar arguments one can also prove that 

m„P„(|e|l^,,_^^^<^J-(-)/(Co)P(|e|) 



[0,ri] 



0. 



0. 



(69) 



(70) 
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Pick a positive number 77 > 0. Taking into account that elco<^<Co+— = iu ~ 
/3o)l(o<z<Co+^ the analogous result for e„ with (n and /3„ instead of Co and 
(3o we see that 



rrir, 



< 



and consequently 



\ \ "S/t ^ .•^11. < 111 "SW ^"SU I in / 

:F,.((|r-ft.|-|i'-^ol)i,„,,,„ii) 



[0,*?] 
[0,»?] 



+ 



VTLn 



< 



Pn(|r-/3o|(l^^ 

- /3o|m 



+ 



+ 



(71) 

We will show that each of the terms on the right-hand side of (71) goes to zero in 
probability. Since n{C,n — Co) = Op(l), we know that for any 5 > there is 
such that P ^n|Cn — Co| > -Rfjj < ^- Then, 



?5 > 



P m„ 



Pn(|F-/3o|(l^„ 



{■) 



^Co<^<Co + ;^ 



>(5 < 5+ 



P ( m„P„ ( |e|l^^_H,^^<^^^ ) > - 1 + 



P I m. 



P (^m„|ao-/3o|P„ (l 



(5 

[0/^] 3 
> 



Co-^<z<CoJ ' 3 



but from equations (69) and (70), and the fact that ^ — )■ 0, we actually get that 
all the terms of the right-hand side are asymptotically smaller than |. Thus, 



lim P I m 



(■) 



■'■Co<^<Co + ;^ 



P"(l^-/5ol(lc„<z<c„ 
An argument similar in spirit to the one just employed gives 



[0/'?] 



>6] < 25. (72) 



lim P I m 



P. 



(n<Z<(n + :^ 



'^<:o<z<(;o+^ 



>5) <5 (73) 
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while equation (74), for ^ = 0, and the strong consistency of the least squares 
estimator give 



0. 



[o,v] 



Then, combining the last identity with (71), (72) and (73) we get 



lim lim m„P 

(5— >0 n— >oo 



Completely analogous arguments prove that 



(■) 



> 6 



[o,v] 



0. 



rrin 



Pn(|e„|l; 



"VFn|-C„_±I<Z<cJ Pn(|e|lco-ii<Z<Co' 



[0,V] 



0. 



Since rj > was arbitrarily chosen, we have shown (IV) for ■?/;(■) = | • |. The case 
= I • P is proven in a very similar manner. For the sake of brevity, we omit the 
proof. 

Now, we consider the case where iplx) = e*^^ for some ^ G M. Again, fix 77 > 0. 
We will proceed in the same way as before. Let t G M and write 

Pn = nF (e^^*^-(^^) - 1 - ^tcos(ee)) P(lco<z<Co+^)- 

Then, < t^^mnF{l(a<z<(o+-^) ~^ 0. The characteristic function of 
m„P„(cos(^e) l^„<2<_|_j_) can be written as 



TTh V 
iPnit) = (l + ^-r^nCOs(e6))P(l^„<z<Co+^) + -7 



e 



itr,P(cos(^e))/{Co) 



and therefore 



m„P„(cos (^e) 1 



Co<^<Co + ; 



r7/(Co)P(cos(ee)). 



Applying the same arguments to the function sin (,^e) we obtain that 



(sin (^e) lco<z<Co+- 



r7/(Co)P(sin(ee)). 



and hence 



r//(Co)^e = ^/(Co)P(e 



The same tightness argument that was applied to prove (69) can be used here to 
conclude that 



m„P„(e^«nCo<z<Co+^) - (■)/(Co)P(e 







(74) 
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and similarly 



Co- — <^<Co^ 



■)/(Co)P(e*^^: 



Using the triangular inequality together with the definition of e„ we get 



< 



[0/'?] 



rrir 



+ 



rrir, 



Co<^<Co- 



But (70) implies that 







while (74) applied when ^ = and the strong consistency of /3„ yield 

< 



rrir 



- (3o\m, 



(•) 



0. 



Therefore, 



rrir. 



which together with (74) proves that 

|m„P„(e''«-l^^^^<^^^i^) - (■)/(Co)P(e^^^) 
With completely analogous arguments one shows 

(■)/(Co)P(e^^- 



[0,ri] 



rrir 



0. 



(75) 



This proves that (VI) holds in probability. 

We now proceed to prove that (V) and (VII) hold in probability. Before embarking 
in this task, we want to make the following remark. Consider that class of func- 
tions C := {elj{z) : / C M is an interval}. Then, this class has a square integrable 
envelope |e| and F{iIj) = for any ip & C. Therefore, the maximal inequality 3.1 
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from Kim and Pollard (1990) implies that ||Pn||c = Op (^n 2^. Similar observa- 



tions also show that ||P„ 



Op in 2 ) • All these considerations, in addition 



with Corollary 3.1, (65), (66), (63) and (64) show that 



/m^0n - /3o) 
mniCn - Co) 



^Tflfi 1 1 1 1 Q 



/nir. 



Pn(|e|l 



Co- 



1± 



{■) 



P 
P 
P 
P 
P 



"rrtr. 



Pn(|en|l 















(76) 
(77) 
(78) 
(79) 
(80) 

(81) 

(82) 



for any compact set C M. 

Let ?7 > be fixed. Take any subsequence (n^)^^ and find a further subsequence 
(^fcs)^i such that all the statements in the previous display happen almost surely 
with the compact set K taken to be i^' = [Co — '^ViCo + 2//]. Now, for such a 
subsequence, there is G N such that |Co — Cn^^ | < 77 V s > A^. Then, for any 
5 > and s > N, the following inequalities are true 



sup <; |r„^^(?„,^l^/,; <z<(vU )l f ^ ~ "ol + 1^"*. ~ ^ol + 



^nk,{\(^nkJ'^^ 2_<2<^„ +^2_) + Pnfc,(|e|li 



Co-^<z<Co+-. 



2v 



riks lie 



■|P. 



2_ 



<Z<(, 



"fes lie 



These last inequalities together with (76)-(82) imply that 



lim 



lim y/TUn^ sup 

™ '|Cn,-CI<52 



^rifc^ sup 1 l^™fc^(enfc^l^ACn. <2<CVCut. 



a.s. 
a.s. 



The previous equations show that (6) and (7) in (V) as well as (VII) hold with 
probability one for the subsequence (^ifcj^i- We conclude by noting that if k = 
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inf {f{z)}, then the mean value theorem imphes 

ze[a,b] 



which in consequence shows 

hm inf <— — \ — rP„, (1^.; ) \ > K > a. s. 

This finishes the proof. □ 
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